v0.3.4

Add sleep tip about ssh

.travis.yml

@@ -0,0 +1,24 @@
+language: go
+
+go:
+    - 1.1
+    - tip
+
+install: make deps
+script:
+    - go test ./...
+    - go test -race ./...
+
+notifications:
+    flowdock:
+        secure: fZrcf9rlh2IrQrlch1sHkn3YI7SKvjGnAl/zyV5D6NROe1Bbr6d3QRMuCXWWdhJHzjKmXk5rIzbqJhUc0PNF7YjxGNKSzqWMQ56KcvN1k8DzlqxpqkcA3Jbs6fXCWo2fssRtZ7hj/wOP1f5n6cc7kzHDt9dgaYJ6nO2fqNPJiTc=
+
+    irc:
+        channels:
+            - "chat.freenode.net#packer-tool"
+        on_success: change
+        on_failure: always
+
+matrix:
+    allow_failures:
+        - go: tip

CHANGELOG.md

@@ -1,3 +1,99 @@
+## 0.3.4 (August 21, 2013)
+
+IMPROVEMENTS:
+
+* post-processor/vagrant: the file being compressed will be shown
+  in the UI [GH-314]
+
+BUG FIXES:
+
+* core: Avoid panics when double-interrupting Packer.
+* provisioner/shell: Retry shell script uploads, making reboots more
+  robust if they happen to fail in this stage. [GH-282]
+
+## 0.3.3 (August 19, 2013)
+
+FEATURES:
+
+* builder/virtualbox: support exporting in OVA format. [GH-309]
+
+IMPROVEMENTS:
+
+* core: All HTTP downloads across Packer now support the standard
+  proxy environmental variables (`HTTP_PROXY`, `NO_PROXY`, etc.) [GH-252]
+* builder/amazon: API requests will use HTTP proxy if specified by
+  enviromental variables.
+* builder/digitalocean: API requests will use HTTP proxy if specified
+  by environmental variables.
+
+BUG FIXES:
+
+* core: TCP connection between plugin processes will keep-alive. [GH-312]
+* core: No more "unused key keep_input_artifact" for post processors [GH-310]
+* post-processor/vagrant: `output_path` templates now work again.
+
+## 0.3.2 (August 18, 2013)
+
+FEATURES:
+
+* New command: `packer inspect`. This command tells you the components of
+  a template. It respects the `-machine-readable` flag as well so you can
+  parse out components of a template.
+* Packer will detect its own crashes (always a bug) and save a "crash.log"
+  file.
+* builder/virtualbox: You may now specify multiple URLs for an ISO
+  using "iso_url" in a template. The URLs will be tried in order.
+* builder/vmware: You may now specify multiple URLs for an ISO
+  using "iso_url" in a template. The URLs will be tried in order.
+
+IMPROVEMENTS:
+
+* core: built with Go 1.1.2
+* core: packer help output now loads much faster.
+* builder/virtualbox: guest_additions_url can now use the `Version`
+  variable to get the VirtualBox version. [GH-272]
+* builder/virtualbox: Do not check for VirtualBox as part of template
+  validation; only check at execution.
+* builder/vmware: Do not check for VMware as part of template validation;
+  only check at execution.
+* command/build: A path of "-" will read the template from stdin.
+* builder/amazon: add block device mappings [GH-90]
+
+BUG FIXES:
+
+* windows: file URLs are easier to get right as Packer
+  has better parsing and error handling for Windows file paths. [GH-284]
+* builder/amazon/all: Modifying more than one AMI attribute type no longer
+  crashes.
+* builder/amazon-instance: send IAM instance profile data. [GH-294]
+* builder/digitalocean: API request parameters are properly URL
+  encoded. [GH-281]
+* builder/virtualbox: dowload progress won't be shown until download
+  actually starts. [GH-288]
+* builder/virtualbox: floppy files names of 13 characters are now properly
+  written to the FAT12 filesystem. [GH-285]
+* builder/vmware: dowload progress won't be shown until download
+  actually starts. [GH-288]
+* builder/vmware: interrupt works while typing commands over VNC.
+* builder/virtualbox: floppy files names of 13 characters are now properly
+  written to the FAT12 filesystem. [GH-285]
+* post-processor/vagrant: Process user variables. [GH-295]
+
+## 0.3.1 (August 12, 2013)
+
+IMPROVEMENTS:
+
+* provisioner/shell: New setting `start_retry_timeout` which is the timeout
+  for the provisioner to attempt to _start_ the remote process. This allows
+  the shell provisioner to work properly with reboots. [GH-260]
+
+BUG FIXES:
+
+* core: Remote command output containing '\r' now looks much better
+  within the Packer output.
+* builder/vmware: Fix issue with finding driver files. [GH-279]
+* provisioner/salt-masterless: Uploads work properly from Windows. [GH-276]
+
 ## 0.3.0 (August 12, 2013)
 
 BACKWARDS INCOMPATIBILITIES:

Makefile

@@ -3,22 +3,24 @@ OK_COLOR=\033[32;01m
 ERROR_COLOR=\033[31;01m
 WARN_COLOR=\033[33;01m
 
-all:
+all: deps
 	@mkdir -p bin/
-	@echo "$(OK_COLOR)==> Installing dependencies$(NO_COLOR)"
-	@go get -d -v ./...
 	@echo "$(OK_COLOR)==> Building$(NO_COLOR)"
 	@./scripts/build.sh
 
+deps:
+	@echo "$(OK_COLOR)==> Installing dependencies$(NO_COLOR)"
+	@go get -d -v ./...
+	@go list -f '{{range .TestImports}}{{.}} {{end}}' ./... | xargs -n1 go get -d
+
 clean:
 	@rm -rf bin/ local/ pkg/ src/ website/.sass-cache website/build
 
 format:
 	go fmt ./...
 
-test:
+test: deps
 	@echo "$(OK_COLOR)==> Testing Packer...$(NO_COLOR)"
-	@go list -f '{{range .TestImports}}{{.}} {{end}}' ./... | xargs -n1 go get -d
 	go test ./...
 
-.PHONY: all format test
+.PHONY: all deps format test

builder/amazon/chroot/builder.go

@@ -34,7 +34,7 @@ type Config struct {
 	SourceAmi      string     `mapstructure:"source_ami"`
 	UnmountCommand string     `mapstructure:"unmount_command"`
 
-	tpl *common.Template
+	tpl *packer.ConfigTemplate
 }
 
 type Builder struct {
@@ -48,7 +48,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	b.config.tpl, err = common.NewTemplate()
+	b.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}

builder/amazon/common/access_config.go

@@ -3,7 +3,7 @@ package common
 import (
 	"fmt"
 	"github.com/mitchellh/goamz/aws"
-	"github.com/mitchellh/packer/common"
+	"github.com/mitchellh/packer/packer"
 	"strings"
 	"unicode"
 )
@@ -37,10 +37,10 @@ func (c *AccessConfig) Region() (aws.Region, error) {
 	return aws.Regions[region], nil
 }
 
-func (c *AccessConfig) Prepare(t *common.Template) []error {
+func (c *AccessConfig) Prepare(t *packer.ConfigTemplate) []error {
 	if t == nil {
 		var err error
-		t, err = common.NewTemplate()
+		t, err = packer.NewConfigTemplate()
 		if err != nil {
 			return []error{err}
 		}

builder/amazon/common/ami_config.go

@@ -2,7 +2,7 @@ package common
 
 import (
 	"fmt"
-	"github.com/mitchellh/packer/common"
+	"github.com/mitchellh/packer/packer"
 )
 
 // AMIConfig is for common configuration related to creating AMIs.
@@ -14,10 +14,10 @@ type AMIConfig struct {
 	AMIProductCodes []string `mapstructure:"ami_product_codes"`
 }
 
-func (c *AMIConfig) Prepare(t *common.Template) []error {
+func (c *AMIConfig) Prepare(t *packer.ConfigTemplate) []error {
 	if t == nil {
 		var err error
-		t, err = common.NewTemplate()
+		t, err = packer.NewConfigTemplate()
 		if err != nil {
 			return []error{err}
 		}

builder/amazon/common/block_device.go

@@ -0,0 +1,46 @@
+package common
+
+import (
+	"github.com/mitchellh/goamz/ec2"
+)
+
+// BlockDevice
+type BlockDevice struct {
+	DeviceName          string `mapstructure:"device_name"`
+	VirtualName         string `mapstructure:"virtual_name"`
+	SnapshotId          string `mapstructure:"snapshot_id"`
+	VolumeType          string `mapstructure:"volume_type"`
+	VolumeSize          int64  `mapstructure:"volume_size"`
+	DeleteOnTermination bool   `mapstructure:"delete_on_termination"`
+	IOPS                int64  `mapstructure:"iops"`
+}
+
+type BlockDevices struct {
+	AMIMappings    []BlockDevice `mapstructure:"ami_block_device_mappings,squash"`
+	LaunchMappings []BlockDevice `mapstructure:"launch_block_device_mappings,squash"`
+}
+
+func buildBlockDevices(b []BlockDevice) []ec2.BlockDeviceMapping {
+	var blockDevices []ec2.BlockDeviceMapping
+
+	for _, blockDevice := range b {
+		blockDevices = append(blockDevices, ec2.BlockDeviceMapping{
+			DeviceName:          blockDevice.DeviceName,
+			VirtualName:         blockDevice.VirtualName,
+			SnapshotId:          blockDevice.SnapshotId,
+			VolumeType:          blockDevice.VolumeType,
+			VolumeSize:          blockDevice.VolumeSize,
+			DeleteOnTermination: blockDevice.DeleteOnTermination,
+			IOPS:                blockDevice.IOPS,
+		})
+	}
+	return blockDevices
+}
+
+func (b *BlockDevices) BuildAMIDevices() []ec2.BlockDeviceMapping {
+	return buildBlockDevices(b.AMIMappings)
+}
+
+func (b *BlockDevices) BuildLaunchDevices() []ec2.BlockDeviceMapping {
+	return buildBlockDevices(b.LaunchMappings)
+}

builder/amazon/common/block_device_test.go

@@ -0,0 +1,41 @@
+package common
+
+import (
+	"cgl.tideland.biz/asserts"
+	"github.com/mitchellh/goamz/ec2"
+	"testing"
+)
+
+func TestBlockDevice(t *testing.T) {
+	assert := asserts.NewTestingAsserts(t, true)
+
+	ec2Mapping := []ec2.BlockDeviceMapping{
+		ec2.BlockDeviceMapping{
+			DeviceName:          "/dev/sdb",
+			VirtualName:         "ephemeral0",
+			SnapshotId:          "snap-1234",
+			VolumeType:          "standard",
+			VolumeSize:          8,
+			DeleteOnTermination: true,
+			IOPS:                1000,
+		},
+	}
+
+	blockDevice := BlockDevice{
+		DeviceName:          "/dev/sdb",
+		VirtualName:         "ephemeral0",
+		SnapshotId:          "snap-1234",
+		VolumeType:          "standard",
+		VolumeSize:          8,
+		DeleteOnTermination: true,
+		IOPS:                1000,
+	}
+
+	blockDevices := BlockDevices{
+		AMIMappings:    []BlockDevice{blockDevice},
+		LaunchMappings: []BlockDevice{blockDevice},
+	}
+
+	assert.Equal(ec2Mapping, blockDevices.BuildAMIDevices(), "should match output")
+	assert.Equal(ec2Mapping, blockDevices.BuildLaunchDevices(), "should match output")
+}

builder/amazon/common/run_config.go

@@ -3,7 +3,7 @@ package common
 import (
 	"errors"
 	"fmt"
-	"github.com/mitchellh/packer/common"
+	"github.com/mitchellh/packer/packer"
 	"os"
 	"time"
 )
@@ -27,10 +27,10 @@ type RunConfig struct {
 	sshTimeout time.Duration
 }
 
-func (c *RunConfig) Prepare(t *common.Template) []error {
+func (c *RunConfig) Prepare(t *packer.ConfigTemplate) []error {
 	if t == nil {
 		var err error
-		t, err = common.NewTemplate()
+		t, err = packer.NewConfigTemplate()
 		if err != nil {
 			return []error{err}
 		}

builder/amazon/common/step_modify_ami_attributes.go

@@ -31,20 +31,44 @@ func (s *StepModifyAMIAttributes) Run(state map[string]interface{}) multistep.St
 		return multistep.ActionContinue
 	}
 
-	options := &ec2.ModifyImageAttribute{
-		Description:  s.Description,
-		AddUsers:     s.Users,
-		AddGroups:    s.Groups,
-		ProductCodes: s.ProductCodes,
+	// Construct the modify image attribute requests we're going to make.
+	// We need to make each separately since the EC2 API only allows changing
+	// one type at a kind currently.
+	options := make(map[string]*ec2.ModifyImageAttribute)
+	if s.Description != "" {
+		options["description"] = &ec2.ModifyImageAttribute{
+			Description: s.Description,
+		}
+	}
+
+	if len(s.Groups) > 0 {
+		options["groups"] = &ec2.ModifyImageAttribute{
+			AddGroups: s.Groups,
+		}
+	}
+
+	if len(s.Users) > 0 {
+		options["users"] = &ec2.ModifyImageAttribute{
+			AddUsers: s.Users,
+		}
+	}
+
+	if len(s.ProductCodes) > 0 {
+		options["product codes"] = &ec2.ModifyImageAttribute{
+			ProductCodes: s.ProductCodes,
+		}
 	}
 
 	ui.Say("Modifying AMI attributes...")
-	_, err := ec2conn.ModifyImageAttribute(ami, options)
-	if err != nil {
-		err := fmt.Errorf("Error modify AMI attributes: %s", err)
-		state["error"] = err
-		ui.Error(err.Error())
-		return multistep.ActionHalt
+	for name, opts := range options {
+		ui.Message(fmt.Sprintf("Modifying: %s", name))
+		_, err := ec2conn.ModifyImageAttribute(ami, opts)
+		if err != nil {
+			err := fmt.Errorf("Error modify AMI attributes: %s", err)
+			state["error"] = err
+			ui.Error(err.Error())
+			return multistep.ActionHalt
+		}
 	}
 
 	return multistep.ActionContinue

builder/amazon/common/step_run_source_instance.go

@@ -17,6 +17,7 @@ type StepRunSourceInstance struct {
 	SourceAMI          string
 	IamInstanceProfile string
 	SubnetId           string
+	BlockDevices       BlockDevices
 
 	instance *ec2.Instance
 }
@@ -48,6 +49,7 @@ func (s *StepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
 		SecurityGroups:     []ec2.SecurityGroup{ec2.SecurityGroup{Id: securityGroupId}},
 		IamInstanceProfile: s.IamInstanceProfile,
 		SubnetId:           s.SubnetId,
+		BlockDevices:       s.BlockDevices.BuildLaunchDevices(),
 	}
 
 	ui.Say("Launching a source AWS instance...")
@@ -90,7 +92,6 @@ func (s *StepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
 		StepState: state,
 	}
 	latestInstance, err := WaitForState(&stateChange)
-	s.instance = latestInstance.(*ec2.Instance)
 	if err != nil {
 		err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", s.instance.InstanceId, err)
 		state["error"] = err
@@ -98,6 +99,7 @@ func (s *StepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
 		return multistep.ActionHalt
 	}
 
+	s.instance = latestInstance.(*ec2.Instance)
 	state["instance"] = s.instance
 
 	return multistep.ActionContinue

builder/amazon/ebs/builder.go

@@ -22,12 +22,13 @@ type config struct {
 	common.PackerConfig    `mapstructure:",squash"`
 	awscommon.AccessConfig `mapstructure:",squash"`
 	awscommon.AMIConfig    `mapstructure:",squash"`
+	awscommon.BlockDevices `mapstructure:",squash"`
 	awscommon.RunConfig    `mapstructure:",squash"`
 
 	// Tags for the AMI
 	Tags map[string]string
 
-	tpl *common.Template
+	tpl *packer.ConfigTemplate
 }
 
 type Builder struct {
@@ -41,7 +42,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	b.config.tpl, err = common.NewTemplate()
+	b.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}
@@ -119,6 +120,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SourceAMI:          b.config.SourceAmi,
 			IamInstanceProfile: b.config.IamInstanceProfile,
 			SubnetId:           b.config.SubnetId,
+			BlockDevices:       b.config.BlockDevices,
 		},
 		&common.StepConnectSSH{
 			SSHAddress:     awscommon.SSHAddress(ec2conn, b.config.SSHPort),

builder/amazon/ebs/step_create_ami.go

@@ -19,8 +19,9 @@ func (s *stepCreateAMI) Run(state map[string]interface{}) multistep.StepAction {
 	// Create the image
 	ui.Say(fmt.Sprintf("Creating the AMI: %s", config.AMIName))
 	createOpts := &ec2.CreateImage{
-		InstanceId: instance.InstanceId,
-		Name:       config.AMIName,
+		InstanceId:   instance.InstanceId,
+		Name:         config.AMIName,
+		BlockDevices: config.BlockDevices.BuildAMIDevices(),
 	}
 
 	createResp, err := ec2conn.CreateImage(createOpts)

builder/amazon/instance/builder.go

@@ -24,6 +24,7 @@ type Config struct {
 	common.PackerConfig    `mapstructure:",squash"`
 	awscommon.AccessConfig `mapstructure:",squash"`
 	awscommon.AMIConfig    `mapstructure:",squash"`
+	awscommon.BlockDevices `mapstructure:",squash"`
 	awscommon.RunConfig    `mapstructure:",squash"`
 
 	AccountId           string `mapstructure:"account_id"`
@@ -37,7 +38,7 @@ type Config struct {
 	X509KeyPath         string `mapstructure:"x509_key_path"`
 	X509UploadPath      string `mapstructure:"x509_upload_path"`
 
-	tpl *common.Template
+	tpl *packer.ConfigTemplate
 }
 
 type Builder struct {
@@ -51,7 +52,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	b.config.tpl, err = common.NewTemplate()
+	b.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}
@@ -193,10 +194,12 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		&awscommon.StepRunSourceInstance{
 			ExpectedRootDevice: "instance-store",
 			InstanceType:       b.config.InstanceType,
+			IamInstanceProfile: b.config.IamInstanceProfile,
 			UserData:           b.config.UserData,
 			UserDataFile:       b.config.UserDataFile,
 			SourceAMI:          b.config.SourceAmi,
 			SubnetId:           b.config.SubnetId,
+			BlockDevices:       b.config.BlockDevices,
 		},
 		&common.StepConnectSSH{
 			SSHAddress:     awscommon.SSHAddress(ec2conn, b.config.SSHPort),

builder/amazon/instance/step_register_ami.go

@@ -20,6 +20,7 @@ func (s *StepRegisterAMI) Run(state map[string]interface{}) multistep.StepAction
 	registerOpts := &ec2.RegisterImage{
 		ImageLocation: manifestPath,
 		Name:          config.AMIName,
+		BlockDevices:  config.BlockDevices.BuildAMIDevices(),
 	}
 
 	registerResp, err := ec2conn.RegisterImage(registerOpts)

builder/digitalocean/api.go

@@ -43,7 +43,11 @@ type DigitalOceanClient struct {
 // Creates a new client for communicating with DO
 func (d DigitalOceanClient) New(client string, key string) *DigitalOceanClient {
 	c := &DigitalOceanClient{
-		client:   http.DefaultClient,
+		client: &http.Client{
+			Transport: &http.Transport{
+				Proxy: http.ProxyFromEnvironment,
+			},
+		},
 		BaseURL:  DIGITALOCEAN_API_URL,
 		ClientID: client,
 		APIKey:   key,
@@ -53,10 +57,9 @@ func (d DigitalOceanClient) New(client string, key string) *DigitalOceanClient {
 
 // Creates an SSH Key and returns it's id
 func (d DigitalOceanClient) CreateKey(name string, pub string) (uint, error) {
-	// Escape the public key
-	pub = url.QueryEscape(pub)
-
-	params := fmt.Sprintf("name=%v&ssh_pub_key=%v", name, pub)
+	params := url.Values{}
+	params.Set("name", name)
+	params.Set("ssh_pub_key", pub)
 
 	body, err := NewRequest(d, "ssh_keys/new", params)
 	if err != nil {
@@ -72,15 +75,18 @@ func (d DigitalOceanClient) CreateKey(name string, pub string) (uint, error) {
 // Destroys an SSH key
 func (d DigitalOceanClient) DestroyKey(id uint) error {
 	path := fmt.Sprintf("ssh_keys/%v/destroy", id)
-	_, err := NewRequest(d, path, "")
+	_, err := NewRequest(d, path, url.Values{})
 	return err
 }
 
 // Creates a droplet and returns it's id
 func (d DigitalOceanClient) CreateDroplet(name string, size uint, image uint, region uint, keyId uint) (uint, error) {
-	params := fmt.Sprintf(
-		"name=%v&image_id=%v&size_id=%v&region_id=%v&ssh_key_ids=%v",
-		name, image, size, region, keyId)
+	params := url.Values{}
+	params.Set("name", name)
+	params.Set("size_id", fmt.Sprintf("%v", size))
+	params.Set("image_id", fmt.Sprintf("%v", image))
+	params.Set("region_id", fmt.Sprintf("%v", region))
+	params.Set("ssh_key_ids", fmt.Sprintf("%v", keyId))
 
 	body, err := NewRequest(d, "droplets/new", params)
 	if err != nil {
@@ -96,7 +102,7 @@ func (d DigitalOceanClient) CreateDroplet(name string, size uint, image uint, re
 // Destroys a droplet
 func (d DigitalOceanClient) DestroyDroplet(id uint) error {
 	path := fmt.Sprintf("droplets/%v/destroy", id)
-	_, err := NewRequest(d, path, "")
+	_, err := NewRequest(d, path, url.Values{})
 	return err
 }
 
@@ -104,7 +110,7 @@ func (d DigitalOceanClient) DestroyDroplet(id uint) error {
 func (d DigitalOceanClient) PowerOffDroplet(id uint) error {
 	path := fmt.Sprintf("droplets/%v/power_off", id)
 
-	_, err := NewRequest(d, path, "")
+	_, err := NewRequest(d, path, url.Values{})
 
 	return err
 }
@@ -112,7 +118,9 @@ func (d DigitalOceanClient) PowerOffDroplet(id uint) error {
 // Creates a snaphot of a droplet by it's ID
 func (d DigitalOceanClient) CreateSnapshot(id uint, name string) error {
 	path := fmt.Sprintf("droplets/%v/snapshot", id)
-	params := fmt.Sprintf("name=%v", name)
+
+	params := url.Values{}
+	params.Set("name", name)
 
 	_, err := NewRequest(d, path, params)
 
@@ -121,7 +129,7 @@ func (d DigitalOceanClient) CreateSnapshot(id uint, name string) error {
 
 // Returns all available images.
 func (d DigitalOceanClient) Images() ([]Image, error) {
-	resp, err := NewRequest(d, "images", "")
+	resp, err := NewRequest(d, "images", url.Values{})
 	if err != nil {
 		return nil, err
 	}
@@ -137,7 +145,7 @@ func (d DigitalOceanClient) Images() ([]Image, error) {
 // Destroys an image by its ID.
 func (d DigitalOceanClient) DestroyImage(id uint) error {
 	path := fmt.Sprintf("images/%d/destroy", id)
-	_, err := NewRequest(d, path, "")
+	_, err := NewRequest(d, path, url.Values{})
 	return err
 }
 
@@ -145,7 +153,7 @@ func (d DigitalOceanClient) DestroyImage(id uint) error {
 func (d DigitalOceanClient) DropletStatus(id uint) (string, string, error) {
 	path := fmt.Sprintf("droplets/%v", id)
 
-	body, err := NewRequest(d, path, "")
+	body, err := NewRequest(d, path, url.Values{})
 	if err != nil {
 		return "", "", err
 	}
@@ -165,10 +173,14 @@ func (d DigitalOceanClient) DropletStatus(id uint) (string, string, error) {
 
 // Sends an api request and returns a generic map[string]interface of
 // the response.
-func NewRequest(d DigitalOceanClient, path string, params string) (map[string]interface{}, error) {
+func NewRequest(d DigitalOceanClient, path string, params url.Values) (map[string]interface{}, error) {
 	client := d.client
-	url := fmt.Sprintf("%s/%s?%s&client_id=%s&api_key=%s",
-		DIGITALOCEAN_API_URL, path, params, d.ClientID, d.APIKey)
+
+	// Add the authentication parameters
+	params.Set("client_id", d.ClientID)
+	params.Set("api_key", d.APIKey)
+
+	url := fmt.Sprintf("%s/%s?%s", DIGITALOCEAN_API_URL, path, params.Encode())
 
 	var decodedResponse map[string]interface{}
 

builder/digitalocean/builder.go

@@ -43,7 +43,7 @@ type config struct {
 	eventDelay   time.Duration
 	stateTimeout time.Duration
 
-	tpl *common.Template
+	tpl *packer.ConfigTemplate
 }
 
 type Builder struct {
@@ -57,7 +57,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	b.config.tpl, err = common.NewTemplate()
+	b.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}

builder/virtualbox/builder.go

@@ -18,7 +18,6 @@ const BuilderId = "mitchellh.virtualbox"
 
 type Builder struct {
 	config config
-	driver Driver
 	runner multistep.Runner
 }
 
@@ -28,6 +27,7 @@ type config struct {
 	BootCommand          []string   `mapstructure:"boot_command"`
 	DiskSize             uint       `mapstructure:"disk_size"`
 	FloppyFiles          []string   `mapstructure:"floppy_files"`
+	Format               string     `mapstructure:"format"`
 	GuestAdditionsPath   string     `mapstructure:"guest_additions_path"`
 	GuestAdditionsURL    string     `mapstructure:"guest_additions_url"`
 	GuestAdditionsSHA256 string     `mapstructure:"guest_additions_sha256"`
@@ -38,7 +38,7 @@ type config struct {
 	HTTPPortMax          uint       `mapstructure:"http_port_max"`
 	ISOChecksum          string     `mapstructure:"iso_checksum"`
 	ISOChecksumType      string     `mapstructure:"iso_checksum_type"`
-	ISOUrl               string     `mapstructure:"iso_url"`
+	ISOUrls              []string   `mapstructure:"iso_urls"`
 	OutputDir            string     `mapstructure:"output_directory"`
 	ShutdownCommand      string     `mapstructure:"shutdown_command"`
 	SSHHostPortMin       uint       `mapstructure:"ssh_host_port_min"`
@@ -51,13 +51,14 @@ type config struct {
 	VMName               string     `mapstructure:"vm_name"`
 
 	RawBootWait        string `mapstructure:"boot_wait"`
+	RawSingleISOUrl    string `mapstructure:"iso_url"`
 	RawShutdownTimeout string `mapstructure:"shutdown_timeout"`
 	RawSSHWaitTimeout  string `mapstructure:"ssh_wait_timeout"`
 
 	bootWait        time.Duration ``
 	shutdownTimeout time.Duration ``
 	sshWaitTimeout  time.Duration ``
-	tpl             *common.Template
+	tpl             *packer.ConfigTemplate
 }
 
 func (b *Builder) Prepare(raws ...interface{}) error {
@@ -66,7 +67,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	b.config.tpl, err = common.NewTemplate()
+	b.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}
@@ -131,22 +132,25 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.VMName = fmt.Sprintf("packer-%s", b.config.PackerBuildName)
 	}
 
+	if b.config.Format == "" {
+		b.config.Format = "ovf"
+	}
+
 	// Errors
 	templates := map[string]*string{
-		"guest_additions_path":    &b.config.GuestAdditionsPath,
-		"guest_additions_url":     &b.config.GuestAdditionsURL,
 		"guest_additions_sha256":  &b.config.GuestAdditionsSHA256,
 		"guest_os_type":           &b.config.GuestOSType,
 		"http_directory":          &b.config.HTTPDir,
 		"iso_checksum":            &b.config.ISOChecksum,
 		"iso_checksum_type":       &b.config.ISOChecksumType,
-		"iso_url":                 &b.config.ISOUrl,
+		"iso_url":                 &b.config.RawSingleISOUrl,
 		"output_directory":        &b.config.OutputDir,
 		"shutdown_command":        &b.config.ShutdownCommand,
 		"ssh_password":            &b.config.SSHPassword,
 		"ssh_username":            &b.config.SSHUser,
 		"virtualbox_version_file": &b.config.VBoxVersionFile,
 		"vm_name":                 &b.config.VMName,
+		"format":                  &b.config.Format,
 		"boot_wait":               &b.config.RawBootWait,
 		"shutdown_timeout":        &b.config.RawShutdownTimeout,
 		"ssh_wait_timeout":        &b.config.RawSSHWaitTimeout,
@@ -161,6 +165,27 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
+	for i, url := range b.config.ISOUrls {
+		var err error
+		b.config.ISOUrls[i], err = b.config.tpl.Process(url, nil)
+		if err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Error processing iso_urls[%d]: %s", i, err))
+		}
+	}
+
+	validates := map[string]*string{
+		"guest_additions_path": &b.config.GuestAdditionsPath,
+		"guest_additions_url":  &b.config.GuestAdditionsURL,
+	}
+
+	for n, ptr := range validates {
+		if err := b.config.tpl.Validate(*ptr); err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Error parsing %s: %s", n, err))
+		}
+	}
+
 	for i, command := range b.config.BootCommand {
 		if err := b.config.tpl.Validate(command); err != nil {
 			errs = packer.MultiErrorAppend(errs,
@@ -178,6 +203,11 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
+	if !(b.config.Format == "ovf" || b.config.Format == "ova") {
+		errs = packer.MultiErrorAppend(
+			errs, errors.New("invalid format, only 'ovf' or 'ova' are allowed"))
+	}
+
 	if b.config.HTTPPortMin > b.config.HTTPPortMax {
 		errs = packer.MultiErrorAppend(
 			errs, errors.New("http_port_min must be less than http_port_max"))
@@ -202,14 +232,21 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
-	if b.config.ISOUrl == "" {
+	if b.config.RawSingleISOUrl == "" && len(b.config.ISOUrls) == 0 {
 		errs = packer.MultiErrorAppend(
-			errs, errors.New("An iso_url must be specified."))
-	} else {
-		b.config.ISOUrl, err = common.DownloadableURL(b.config.ISOUrl)
+			errs, errors.New("One of iso_url or iso_urls must be specified."))
+	} else if b.config.RawSingleISOUrl != "" && len(b.config.ISOUrls) > 0 {
+		errs = packer.MultiErrorAppend(
+			errs, errors.New("Only one of iso_url or iso_urls may be specified."))
+	} else if b.config.RawSingleISOUrl != "" {
+		b.config.ISOUrls = []string{b.config.RawSingleISOUrl}
+	}
+
+	for i, url := range b.config.ISOUrls {
+		b.config.ISOUrls[i], err = common.DownloadableURL(url)
 		if err != nil {
 			errs = packer.MultiErrorAppend(
-				errs, fmt.Errorf("iso_url: %s", err))
+				errs, fmt.Errorf("Failed to parse iso_url %d: %s", i+1, err))
 		}
 	}
 
@@ -217,14 +254,6 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.GuestAdditionsSHA256 = strings.ToLower(b.config.GuestAdditionsSHA256)
 	}
 
-	if b.config.GuestAdditionsURL != "" {
-		b.config.GuestAdditionsURL, err = common.DownloadableURL(b.config.GuestAdditionsURL)
-		if err != nil {
-			errs = packer.MultiErrorAppend(
-				errs, fmt.Errorf("guest_additions_url: %s", err))
-		}
-	}
-
 	if !b.config.PackerForce {
 		if _, err := os.Stat(b.config.OutputDir); err == nil {
 			errs = packer.MultiErrorAppend(
@@ -278,12 +307,6 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
-	b.driver, err = b.newDriver()
-	if err != nil {
-		errs = packer.MultiErrorAppend(
-			errs, fmt.Errorf("Failed creating VirtualBox driver: %s", err))
-	}
-
 	if errs != nil && len(errs.Errors) > 0 {
 		return errs
 	}
@@ -292,9 +315,21 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 }
 
 func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packer.Artifact, error) {
+	// Create the driver that we'll use to communicate with VirtualBox
+	driver, err := b.newDriver()
+	if err != nil {
+		return nil, fmt.Errorf("Failed creating VirtualBox driver: %s", err)
+	}
+
 	steps := []multistep.Step{
 		new(stepDownloadGuestAdditions),
-		new(stepDownloadISO),
+		&common.StepDownload{
+			Checksum:     b.config.ISOChecksum,
+			ChecksumType: b.config.ISOChecksumType,
+			Description:  "ISO",
+			ResultKey:    "iso_path",
+			Url:          b.config.ISOUrls,
+		},
 		new(stepPrepareOutputDir),
 		&common.StepCreateFloppy{
 			Files: b.config.FloppyFiles,
@@ -325,7 +360,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 	state := make(map[string]interface{})
 	state["cache"] = cache
 	state["config"] = &b.config
-	state["driver"] = b.driver
+	state["driver"] = driver
 	state["hook"] = hook
 	state["ui"] = ui
 

builder/virtualbox/builder_test.go

@@ -58,6 +58,10 @@ func TestBuilderPrepare_Defaults(t *testing.T) {
 	if b.config.VMName != "packer-foo" {
 		t.Errorf("bad vm name: %s", b.config.VMName)
 	}
+
+	if b.config.Format != "ovf" {
+		t.Errorf("bad format: %s", b.config.Format)
+	}
 }
 
 func TestBuilderPrepare_BootWait(t *testing.T) {
@@ -84,6 +88,7 @@ func TestBuilderPrepare_BootWait(t *testing.T) {
 
 	// Test with a good one
 	config["boot_wait"] = "5s"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -210,6 +215,7 @@ func TestBuilderPrepare_GuestAdditionsURL(t *testing.T) {
 	}
 
 	config["guest_additions_url"] = "http://www.packer.io"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Errorf("should not have error: %s", err)
@@ -230,6 +236,7 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
 
 	// Bad
 	config["http_port_min"] = -500
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -238,6 +245,35 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
 	// Good
 	config["http_port_min"] = 500
 	config["http_port_max"] = 1000
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+}
+
+func TestBuilderPrepare_Format(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Bad
+	config["format"] = "illegal value"
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Good
+	config["format"] = "ova"
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	// Good
+	config["format"] = "ovf"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -269,6 +305,7 @@ func TestBuilderPrepare_ISOChecksum(t *testing.T) {
 
 	// Test good
 	config["iso_checksum"] = "FOo"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -292,6 +329,7 @@ func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
 
 	// Test good
 	config["iso_checksum_type"] = "mD5"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -303,6 +341,7 @@ func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
 
 	// Test unknown
 	config["iso_checksum_type"] = "fake"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -312,18 +351,59 @@ func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
 func TestBuilderPrepare_ISOUrl(t *testing.T) {
 	var b Builder
 	config := testConfig()
+	delete(config, "iso_url")
+	delete(config, "iso_urls")
 
+	// Test both epty
 	config["iso_url"] = ""
+	b = Builder{}
 	err := b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
 	}
 
+	// Test iso_url set
 	config["iso_url"] = "http://www.packer.io"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Errorf("should not have error: %s", err)
 	}
+
+	expected := []string{"http://www.packer.io"}
+	if !reflect.DeepEqual(b.config.ISOUrls, expected) {
+		t.Fatalf("bad: %#v", b.config.ISOUrls)
+	}
+
+	// Test both set
+	config["iso_url"] = "http://www.packer.io"
+	config["iso_urls"] = []string{"http://www.packer.io"}
+	b = Builder{}
+	err = b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Test just iso_urls set
+	delete(config, "iso_url")
+	config["iso_urls"] = []string{
+		"http://www.packer.io",
+		"http://www.hashicorp.com",
+	}
+
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Errorf("should not have error: %s", err)
+	}
+
+	expected = []string{
+		"http://www.packer.io",
+		"http://www.hashicorp.com",
+	}
+	if !reflect.DeepEqual(b.config.ISOUrls, expected) {
+		t.Fatalf("bad: %#v", b.config.ISOUrls)
+	}
 }
 
 func TestBuilderPrepare_OutputDir(t *testing.T) {
@@ -338,6 +418,7 @@ func TestBuilderPrepare_OutputDir(t *testing.T) {
 	defer os.RemoveAll(dir)
 
 	config["output_directory"] = dir
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -345,6 +426,7 @@ func TestBuilderPrepare_OutputDir(t *testing.T) {
 
 	// Test with a good one
 	config["output_directory"] = "i-hope-i-dont-exist"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -364,6 +446,7 @@ func TestBuilderPrepare_ShutdownTimeout(t *testing.T) {
 
 	// Test with a good one
 	config["shutdown_timeout"] = "5s"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -377,6 +460,7 @@ func TestBuilderPrepare_SSHHostPort(t *testing.T) {
 	// Bad
 	config["ssh_host_port_min"] = 1000
 	config["ssh_host_port_max"] = 500
+	b = Builder{}
 	err := b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -384,6 +468,7 @@ func TestBuilderPrepare_SSHHostPort(t *testing.T) {
 
 	// Bad
 	config["ssh_host_port_min"] = -500
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -392,6 +477,7 @@ func TestBuilderPrepare_SSHHostPort(t *testing.T) {
 	// Good
 	config["ssh_host_port_min"] = 500
 	config["ssh_host_port_max"] = 1000
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -403,12 +489,14 @@ func TestBuilderPrepare_SSHUser(t *testing.T) {
 	config := testConfig()
 
 	config["ssh_username"] = ""
+	b = Builder{}
 	err := b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
 	}
 
 	config["ssh_username"] = "exists"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -432,6 +520,7 @@ func TestBuilderPrepare_SSHWaitTimeout(t *testing.T) {
 
 	// Test with a bad value
 	config["ssh_wait_timeout"] = "this is not good"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -439,6 +528,7 @@ func TestBuilderPrepare_SSHWaitTimeout(t *testing.T) {
 
 	// Test with a good one
 	config["ssh_wait_timeout"] = "5s"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)

builder/virtualbox/step_download_guest_additions.go

@@ -2,8 +2,6 @@ package virtualbox
 
 import (
 	"bytes"
-	"crypto/sha256"
-	"encoding/hex"
 	"fmt"
 	"github.com/mitchellh/multistep"
 	"github.com/mitchellh/packer/common"
@@ -13,7 +11,6 @@ import (
 	"log"
 	"os"
 	"strings"
-	"time"
 )
 
 var additionsVersionMap = map[string]string{
@@ -21,6 +18,10 @@ var additionsVersionMap = map[string]string{
 	"4.1.23": "4.1.22",
 }
 
+type guestAdditionsUrlTemplate struct {
+	Version string
+}
+
 // This step uploads a file containing the VirtualBox version, which
 // can be useful for various provisioning reasons.
 //
@@ -30,7 +31,6 @@ type stepDownloadGuestAdditions struct{}
 
 func (s *stepDownloadGuestAdditions) Run(state map[string]interface{}) multistep.StepAction {
 	var action multistep.StepAction
-	cache := state["cache"].(packer.Cache)
 	driver := state["driver"].(Driver)
 	ui := state["ui"].(packer.Ui)
 	config := state["config"].(*config)
@@ -61,87 +61,56 @@ func (s *stepDownloadGuestAdditions) Run(state map[string]interface{}) multistep
 		}
 	}
 
-	checksumBytes, err := hex.DecodeString(checksum)
-	if err != nil {
-		state["error"] = fmt.Errorf("Couldn't decode checksum into bytes: %s", checksum)
-		return multistep.ActionHalt
-	}
-
 	// Use the provided source (URL or file path) or generate it
 	url := config.GuestAdditionsURL
-	if url == "" {
+	if url != "" {
+		tplData := &guestAdditionsUrlTemplate{
+			Version: version,
+		}
+
+		url, err = config.tpl.Process(url, tplData)
+		if err != nil {
+			err := fmt.Errorf("Error preparing guest additions url: %s", err)
+			state["error"] = err
+			ui.Error(err.Error())
+			return multistep.ActionHalt
+		}
+	} else {
 		url = fmt.Sprintf(
 			"http://download.virtualbox.org/virtualbox/%s/%s",
 			version,
 			additionsName)
 	}
 
-	log.Printf("Guest additions URL: %s", url)
-
-	log.Printf("Acquiring lock to download the guest additions ISO.")
-	cachePath := cache.Lock(url)
-	defer cache.Unlock(url)
-
-	downloadConfig := &common.DownloadConfig{
-		Url:        url,
-		TargetPath: cachePath,
-		Hash:       sha256.New(),
-		Checksum:   checksumBytes,
+	url, err = common.DownloadableURL(url)
+	if err != nil {
+		err := fmt.Errorf("Error preparing guest additions url: %s", err)
+		state["error"] = err
+		ui.Error(err.Error())
+		return multistep.ActionHalt
 	}
 
-	download := common.NewDownloadClient(downloadConfig)
-	ui.Say("Downloading VirtualBox guest additions. Progress will be shown periodically.")
-	state["guest_additions_path"], action = s.progressDownload(download, state)
-	return action
+	log.Printf("Guest additions URL: %s", url)
+
+	downStep := &common.StepDownload{
+		Checksum:     checksum,
+		ChecksumType: "sha256",
+		Description:  "Guest additions",
+		ResultKey:    "guest_additions_path",
+		Url:          []string{url},
+	}
+
+	return downStep.Run(state)
 }
 
 func (s *stepDownloadGuestAdditions) Cleanup(state map[string]interface{}) {}
 
-func (s *stepDownloadGuestAdditions) progressDownload(c *common.DownloadClient, state map[string]interface{}) (string, multistep.StepAction) {
-	ui := state["ui"].(packer.Ui)
-
-	var result string
-	downloadCompleteCh := make(chan error, 1)
-
-	// Start a goroutine to actually do the download...
-	go func() {
-		var err error
-		result, err = c.Get()
-		downloadCompleteCh <- err
-	}()
-
-	progressTicker := time.NewTicker(5 * time.Second)
-	defer progressTicker.Stop()
-
-	// A loop that handles showing progress as well as timing out and handling
-	// interrupts and all that.
-DownloadWaitLoop:
-	for {
-		select {
-		case err := <-downloadCompleteCh:
-			if err != nil {
-				state["error"] = fmt.Errorf("Error downloading: %s", err)
-				return "", multistep.ActionHalt
-			}
-
-			break DownloadWaitLoop
-		case <-progressTicker.C:
-			ui.Message(fmt.Sprintf("Download progress: %d%%", c.PercentProgress()))
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				ui.Say("Interrupt received. Cancelling download...")
-				return "", multistep.ActionHalt
-			}
-		}
-	}
-
-	return result, multistep.ActionContinue
-}
-
 func (s *stepDownloadGuestAdditions) downloadAdditionsSHA256(state map[string]interface{}, additionsVersion string, additionsName string) (string, multistep.StepAction) {
 	// First things first, we get the list of checksums for the files available
 	// for this version.
-	checksumsUrl := fmt.Sprintf("http://download.virtualbox.org/virtualbox/%s/SHA256SUMS", additionsVersion)
+	checksumsUrl := fmt.Sprintf(
+		"http://download.virtualbox.org/virtualbox/%s/SHA256SUMS",
+		additionsVersion)
 
 	checksumsFile, err := ioutil.TempFile("", "packer")
 	if err != nil {
@@ -151,25 +120,23 @@ func (s *stepDownloadGuestAdditions) downloadAdditionsSHA256(state map[string]in
 		return "", multistep.ActionHalt
 	}
 	defer os.Remove(checksumsFile.Name())
-
 	checksumsFile.Close()
 
-	downloadConfig := &common.DownloadConfig{
-		Url:        checksumsUrl,
-		TargetPath: checksumsFile.Name(),
-		Hash:       nil,
+	downStep := &common.StepDownload{
+		Description: "Guest additions checksums",
+		ResultKey:   "guest_additions_checksums_path",
+		TargetPath:  checksumsFile.Name(),
+		Url:         []string{checksumsUrl},
 	}
 
-	log.Printf("Downloading guest addition checksums: %s", checksumsUrl)
-	download := common.NewDownloadClient(downloadConfig)
-	checksumsPath, action := s.progressDownload(download, state)
-	if action != multistep.ActionContinue {
+	action := downStep.Run(state)
+	if action == multistep.ActionHalt {
 		return "", action
 	}
 
 	// Next, we find the checksum for the file we're looking to download.
 	// It is an error if the checksum cannot be found.
-	checksumsF, err := os.Open(checksumsPath)
+	checksumsF, err := os.Open(state["guest_additions_checksums_path"].(string))
 	if err != nil {
 		state["error"] = fmt.Errorf("Error opening guest addition checksums: %s", err)
 		return "", multistep.ActionHalt
@@ -199,7 +166,8 @@ func (s *stepDownloadGuestAdditions) downloadAdditionsSHA256(state map[string]in
 	}
 
 	if checksum == "" {
-		state["error"] = fmt.Errorf("The checksum for the file '%s' could not be found.", additionsName)
+		state["error"] = fmt.Errorf(
+			"The checksum for the file '%s' could not be found.", additionsName)
 		return "", multistep.ActionHalt
 	}
 

builder/virtualbox/step_download_iso.go

@@ -1,87 +0,0 @@
-package virtualbox
-
-import (
-	"encoding/hex"
-	"fmt"
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/common"
-	"github.com/mitchellh/packer/packer"
-	"log"
-	"time"
-)
-
-// This step downloads the ISO specified.
-//
-// Uses:
-//   cache packer.Cache
-//   config *config
-//   ui     packer.Ui
-//
-// Produces:
-//   iso_path string
-type stepDownloadISO struct{}
-
-func (s stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction {
-	cache := state["cache"].(packer.Cache)
-	config := state["config"].(*config)
-	ui := state["ui"].(packer.Ui)
-
-	checksum, err := hex.DecodeString(config.ISOChecksum)
-	if err != nil {
-		state["error"] = fmt.Errorf("Error parsing checksum: %s", err)
-		return multistep.ActionHalt
-	}
-
-	log.Printf("Acquiring lock to download the ISO.")
-	cachePath := cache.Lock(config.ISOUrl)
-	defer cache.Unlock(config.ISOUrl)
-
-	downloadConfig := &common.DownloadConfig{
-		Url:        config.ISOUrl,
-		TargetPath: cachePath,
-		CopyFile:   false,
-		Hash:       common.HashForType(config.ISOChecksumType),
-		Checksum:   checksum,
-	}
-
-	download := common.NewDownloadClient(downloadConfig)
-
-	downloadCompleteCh := make(chan error, 1)
-	go func() {
-		ui.Say("Copying or downloading ISO. Progress will be reported periodically.")
-		cachePath, err = download.Get()
-		downloadCompleteCh <- err
-	}()
-
-	progressTicker := time.NewTicker(5 * time.Second)
-	defer progressTicker.Stop()
-
-DownloadWaitLoop:
-	for {
-		select {
-		case err := <-downloadCompleteCh:
-			if err != nil {
-				err := fmt.Errorf("Error downloading ISO: %s", err)
-				state["error"] = err
-				ui.Error(err.Error())
-				return multistep.ActionHalt
-			}
-
-			break DownloadWaitLoop
-		case <-progressTicker.C:
-			ui.Message(fmt.Sprintf("Download progress: %d%%", download.PercentProgress()))
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				ui.Say("Interrupt received. Cancelling download...")
-				return multistep.ActionHalt
-			}
-		}
-	}
-
-	log.Printf("Path to ISO on disk: %s", cachePath)
-	state["iso_path"] = cachePath
-
-	return multistep.ActionContinue
-}
-
-func (stepDownloadISO) Cleanup(map[string]interface{}) {}

builder/virtualbox/step_export.go

@@ -50,7 +50,7 @@ func (s *stepExport) Run(state map[string]interface{}) multistep.StepAction {
 	}
 
 	// Export the VM to an OVF
-	outputPath := filepath.Join(config.OutputDir, "packer.ovf")
+	outputPath := filepath.Join(config.OutputDir, "packer."+config.Format)
 
 	command = []string{
 		"export",

builder/vmware/builder.go

@@ -19,7 +19,6 @@ const BuilderId = "mitchellh.vmware"
 
 type Builder struct {
 	config config
-	driver Driver
 	runner multistep.Runner
 }
 
@@ -32,7 +31,7 @@ type config struct {
 	GuestOSType       string            `mapstructure:"guest_os_type"`
 	ISOChecksum       string            `mapstructure:"iso_checksum"`
 	ISOChecksumType   string            `mapstructure:"iso_checksum_type"`
-	ISOUrl            string            `mapstructure:"iso_url"`
+	ISOUrls           []string          `mapstructure:"iso_urls"`
 	VMName            string            `mapstructure:"vm_name"`
 	OutputDir         string            `mapstructure:"output_directory"`
 	Headless          bool              `mapstructure:"headless"`
@@ -52,13 +51,14 @@ type config struct {
 	VNCPortMax        uint              `mapstructure:"vnc_port_max"`
 
 	RawBootWait        string `mapstructure:"boot_wait"`
+	RawSingleISOUrl    string `mapstructure:"iso_url"`
 	RawShutdownTimeout string `mapstructure:"shutdown_timeout"`
 	RawSSHWaitTimeout  string `mapstructure:"ssh_wait_timeout"`
 
 	bootWait        time.Duration ``
 	shutdownTimeout time.Duration ``
 	sshWaitTimeout  time.Duration ``
-	tpl             *common.Template
+	tpl             *packer.ConfigTemplate
 }
 
 func (b *Builder) Prepare(raws ...interface{}) error {
@@ -67,7 +67,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	b.config.tpl, err = common.NewTemplate()
+	b.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}
@@ -135,7 +135,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		"http_directory":      &b.config.HTTPDir,
 		"iso_checksum":        &b.config.ISOChecksum,
 		"iso_checksum_type":   &b.config.ISOChecksumType,
-		"iso_url":             &b.config.ISOUrl,
+		"iso_url":             &b.config.RawSingleISOUrl,
 		"output_directory":    &b.config.OutputDir,
 		"shutdown_command":    &b.config.ShutdownCommand,
 		"ssh_password":        &b.config.SSHPassword,
@@ -156,6 +156,15 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
+	for i, url := range b.config.ISOUrls {
+		var err error
+		b.config.ISOUrls[i], err = b.config.tpl.Process(url, nil)
+		if err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Error processing iso_urls[%d]: %s", i, err))
+		}
+	}
+
 	for i, command := range b.config.BootCommand {
 		if err := b.config.tpl.Validate(command); err != nil {
 			errs = packer.MultiErrorAppend(errs,
@@ -218,14 +227,21 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
-	if b.config.ISOUrl == "" {
+	if b.config.RawSingleISOUrl == "" && len(b.config.ISOUrls) == 0 {
 		errs = packer.MultiErrorAppend(
-			errs, errors.New("An iso_url must be specified."))
-	} else {
-		b.config.ISOUrl, err = common.DownloadableURL(b.config.ISOUrl)
+			errs, errors.New("One of iso_url or iso_urls must be specified."))
+	} else if b.config.RawSingleISOUrl != "" && len(b.config.ISOUrls) > 0 {
+		errs = packer.MultiErrorAppend(
+			errs, errors.New("Only one of iso_url or iso_urls may be specified."))
+	} else if b.config.RawSingleISOUrl != "" {
+		b.config.ISOUrls = []string{b.config.RawSingleISOUrl}
+	}
+
+	for i, url := range b.config.ISOUrls {
+		b.config.ISOUrls[i], err = common.DownloadableURL(url)
 		if err != nil {
 			errs = packer.MultiErrorAppend(
-				errs, fmt.Errorf("iso_url: %s", err))
+				errs, fmt.Errorf("Failed to parse iso_url %d: %s", i+1, err))
 		}
 	}
 
@@ -280,12 +296,6 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 			errs, fmt.Errorf("vnc_port_min must be less than vnc_port_max"))
 	}
 
-	b.driver, err = NewDriver()
-	if err != nil {
-		errs = packer.MultiErrorAppend(
-			errs, fmt.Errorf("Failed creating VMware driver: %s", err))
-	}
-
 	if errs != nil && len(errs.Errors) > 0 {
 		return errs
 	}
@@ -294,12 +304,24 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 }
 
 func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packer.Artifact, error) {
+	// Initialize the driver that will handle our interaction with VMware
+	driver, err := NewDriver()
+	if err != nil {
+		return nil, fmt.Errorf("Failed creating VMware driver: %s", err)
+	}
+
 	// Seed the random number generator
 	rand.Seed(time.Now().UTC().UnixNano())
 
 	steps := []multistep.Step{
 		&stepPrepareTools{},
-		&stepDownloadISO{},
+		&common.StepDownload{
+			Checksum:     b.config.ISOChecksum,
+			ChecksumType: b.config.ISOChecksumType,
+			Description:  "ISO",
+			ResultKey:    "iso_path",
+			Url:          b.config.ISOUrls,
+		},
 		&stepPrepareOutputDir{},
 		&common.StepCreateFloppy{
 			Files: b.config.FloppyFiles,
@@ -327,7 +349,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 	state := make(map[string]interface{})
 	state["cache"] = cache
 	state["config"] = &b.config
-	state["driver"] = b.driver
+	state["driver"] = driver
 	state["hook"] = hook
 	state["ui"] = ui
 

builder/vmware/builder_test.go

@@ -52,6 +52,7 @@ func TestBuilderPrepare_BootWait(t *testing.T) {
 
 	// Test with a good one
 	config["boot_wait"] = "5s"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -71,6 +72,7 @@ func TestBuilderPrepare_ISOChecksum(t *testing.T) {
 
 	// Test good
 	config["iso_checksum"] = "FOo"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -94,6 +96,7 @@ func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
 
 	// Test good
 	config["iso_checksum_type"] = "mD5"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -105,6 +108,7 @@ func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
 
 	// Test unknown
 	config["iso_checksum_type"] = "fake"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -202,6 +206,7 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
 
 	// Bad
 	config["http_port_min"] = -500
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -210,6 +215,7 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
 	// Good
 	config["http_port_min"] = 500
 	config["http_port_max"] = 1000
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -231,18 +237,59 @@ func TestBuilderPrepare_InvalidKey(t *testing.T) {
 func TestBuilderPrepare_ISOUrl(t *testing.T) {
 	var b Builder
 	config := testConfig()
+	delete(config, "iso_url")
+	delete(config, "iso_urls")
 
+	// Test both epty
 	config["iso_url"] = ""
+	b = Builder{}
 	err := b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
 	}
 
+	// Test iso_url set
 	config["iso_url"] = "http://www.packer.io"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Errorf("should not have error: %s", err)
 	}
+
+	expected := []string{"http://www.packer.io"}
+	if !reflect.DeepEqual(b.config.ISOUrls, expected) {
+		t.Fatalf("bad: %#v", b.config.ISOUrls)
+	}
+
+	// Test both set
+	config["iso_url"] = "http://www.packer.io"
+	config["iso_urls"] = []string{"http://www.packer.io"}
+	b = Builder{}
+	err = b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Test just iso_urls set
+	delete(config, "iso_url")
+	config["iso_urls"] = []string{
+		"http://www.packer.io",
+		"http://www.hashicorp.com",
+	}
+
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Errorf("should not have error: %s", err)
+	}
+
+	expected = []string{
+		"http://www.packer.io",
+		"http://www.hashicorp.com",
+	}
+	if !reflect.DeepEqual(b.config.ISOUrls, expected) {
+		t.Fatalf("bad: %#v", b.config.ISOUrls)
+	}
 }
 
 func TestBuilderPrepare_OutputDir(t *testing.T) {
@@ -257,6 +304,7 @@ func TestBuilderPrepare_OutputDir(t *testing.T) {
 	defer os.RemoveAll(dir)
 
 	config["output_directory"] = dir
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -264,6 +312,7 @@ func TestBuilderPrepare_OutputDir(t *testing.T) {
 
 	// Test with a good one
 	config["output_directory"] = "i-hope-i-dont-exist"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -283,6 +332,7 @@ func TestBuilderPrepare_ShutdownTimeout(t *testing.T) {
 
 	// Test with a good one
 	config["shutdown_timeout"] = "5s"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -300,6 +350,7 @@ func TestBuilderPrepare_SSHUser(t *testing.T) {
 	}
 
 	config["ssh_username"] = "exists"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -347,6 +398,7 @@ func TestBuilderPrepare_SSHWaitTimeout(t *testing.T) {
 
 	// Test with a good one
 	config["ssh_wait_timeout"] = "5s"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -370,6 +422,7 @@ func TestBuilderPrepare_ToolsUploadPath(t *testing.T) {
 
 	// Test with a bad value
 	config["tools_upload_path"] = "{{{nope}"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -377,6 +430,7 @@ func TestBuilderPrepare_ToolsUploadPath(t *testing.T) {
 
 	// Test with a good one
 	config["tools_upload_path"] = "hey"
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
@@ -397,6 +451,7 @@ func TestBuilderPrepare_VNCPort(t *testing.T) {
 
 	// Bad
 	config["vnc_port_min"] = -500
+	b = Builder{}
 	err = b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
@@ -405,6 +460,7 @@ func TestBuilderPrepare_VNCPort(t *testing.T) {
 	// Good
 	config["vnc_port_min"] = 500
 	config["vnc_port_max"] = 1000
+	b = Builder{}
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)

builder/vmware/driver_workstation9_windows.go

@@ -27,7 +27,7 @@ func workstationFindVdiskManager() (string, error) {
 }
 
 func workstationFindVMware() (string, error) {
-	path, _ := exec.LookPath("vmware.exe")
+	path, err := exec.LookPath("vmware.exe")
 	if err == nil {
 		return path, nil
 	}
@@ -36,7 +36,7 @@ func workstationFindVMware() (string, error) {
 }
 
 func workstationFindVmrun() (string, error) {
-	path, _ := exec.LookPath("vmrun.exe")
+	path, err := exec.LookPath("vmrun.exe")
 	if err == nil {
 		return path, nil
 	}
@@ -45,7 +45,7 @@ func workstationFindVmrun() (string, error) {
 }
 
 func workstationToolsIsoPath(flavor string) string {
-	return findFile(flavor+".iso", workstationProgramFilePaths()), nil
+	return findFile(flavor+".iso", workstationProgramFilePaths())
 }
 
 func workstationDhcpLeasesPath(device string) string {
@@ -56,11 +56,11 @@ func workstationDhcpLeasesPath(device string) string {
 		return path
 	}
 
-	return findFile("vmnetdhcp.leases", workstationDataFilePaths()), nil
+	return findFile("vmnetdhcp.leases", workstationDataFilePaths())
 }
 
 func workstationVmnetnatConfPath() string {
-	return findFile("vmnetnat.conf", workstationDataFilePaths()), nil
+	return findFile("vmnetnat.conf", workstationDataFilePaths())
 }
 
 // See http://blog.natefinch.com/2012/11/go-win-stuff.html
@@ -141,7 +141,7 @@ func findFile(file string, paths []string) string {
 		path = normalizePath(path)
 		log.Printf("Searching for file '%s'", path)
 
-		if _, err := os.Stat(path); err != nil {
+		if _, err := os.Stat(path); err == nil {
 			log.Printf("Found file '%s'", path)
 			return path
 		}
@@ -184,7 +184,7 @@ func workstationProgramFilePaths() []string {
 // workstationDataFilePaths returns a list of paths that are eligible
 // to contain data files we may want such as vmnet NAT configuration files.
 func workstationDataFilePaths() []string {
-	leasesPath, err := workstationVmnetDhcpLeasesPathFromRegistry()
+	leasesPath, err := workstationDhcpLeasesPathRegistry()
 	if err != nil {
 		log.Printf("Error getting DHCP leases path: %s", err)
 	}
@@ -198,7 +198,7 @@ func workstationDataFilePaths() []string {
 		paths = append(paths, os.Getenv("VMWARE_DATA"))
 	}
 
-	if path != "" {
+	if leasesPath != "" {
 		paths = append(paths, leasesPath)
 	}
 

builder/vmware/step_download_iso.go

@@ -1,89 +0,0 @@
-package vmware
-
-import (
-	"encoding/hex"
-	"fmt"
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/common"
-	"github.com/mitchellh/packer/packer"
-	"log"
-	"time"
-)
-
-// This step downloads the ISO specified.
-//
-// Uses:
-//   cache packer.Cache
-//   config *config
-//   ui     packer.Ui
-//
-// Produces:
-//   iso_path string
-type stepDownloadISO struct{}
-
-func (s stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction {
-	cache := state["cache"].(packer.Cache)
-	config := state["config"].(*config)
-	ui := state["ui"].(packer.Ui)
-
-	checksum, err := hex.DecodeString(config.ISOChecksum)
-	if err != nil {
-		err := fmt.Errorf("Error parsing checksum: %s", err)
-		state["error"] = err
-		ui.Error(err.Error())
-		return multistep.ActionHalt
-	}
-
-	log.Printf("Acquiring lock to download the ISO.")
-	cachePath := cache.Lock(config.ISOUrl)
-	defer cache.Unlock(config.ISOUrl)
-
-	downloadConfig := &common.DownloadConfig{
-		Url:        config.ISOUrl,
-		TargetPath: cachePath,
-		CopyFile:   false,
-		Hash:       common.HashForType(config.ISOChecksumType),
-		Checksum:   checksum,
-	}
-
-	download := common.NewDownloadClient(downloadConfig)
-
-	downloadCompleteCh := make(chan error, 1)
-	go func() {
-		ui.Say("Copying or downloading ISO. Progress will be reported periodically.")
-		cachePath, err = download.Get()
-		downloadCompleteCh <- err
-	}()
-
-	progressTicker := time.NewTicker(5 * time.Second)
-	defer progressTicker.Stop()
-
-DownloadWaitLoop:
-	for {
-		select {
-		case err := <-downloadCompleteCh:
-			if err != nil {
-				err := fmt.Errorf("Error downloading ISO: %s", err)
-				state["error"] = err
-				ui.Error(err.Error())
-				return multistep.ActionHalt
-			}
-
-			break DownloadWaitLoop
-		case <-progressTicker.C:
-			ui.Say(fmt.Sprintf("Download progress: %d%%", download.PercentProgress()))
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				ui.Say("Interrupt received. Cancelling download...")
-				return multistep.ActionHalt
-			}
-		}
-	}
-
-	log.Printf("Path to ISO on disk: %s", cachePath)
-	state["iso_path"] = cachePath
-
-	return multistep.ActionContinue
-}
-
-func (stepDownloadISO) Cleanup(map[string]interface{}) {}

builder/vmware/step_type_boot_command.go

@@ -96,6 +96,12 @@ func (s *stepTypeBootCommand) Run(state map[string]interface{}) multistep.StepAc
 			return multistep.ActionHalt
 		}
 
+		// Check for interrupts between typing things so we can cancel
+		// since this isn't the fastest thing.
+		if _, ok := state[multistep.StateCancelled]; ok {
+			return multistep.ActionHalt
+		}
+
 		vncSendString(c, command)
 	}
 

command/inspect/command.go

@@ -0,0 +1,118 @@
+package inspect
+
+import (
+	"flag"
+	"fmt"
+	"github.com/mitchellh/packer/packer"
+	"log"
+	"sort"
+	"strings"
+)
+
+type Command struct{}
+
+func (Command) Help() string {
+	return strings.TrimSpace(helpText)
+}
+
+func (c Command) Synopsis() string {
+	return "see components of a template"
+}
+
+func (c Command) Run(env packer.Environment, args []string) int {
+	flags := flag.NewFlagSet("inspect", flag.ContinueOnError)
+	flags.Usage = func() { env.Ui().Say(c.Help()) }
+	if err := flags.Parse(args); err != nil {
+		return 1
+	}
+
+	args = flags.Args()
+	if len(args) != 1 {
+		flags.Usage()
+		return 1
+	}
+
+	// Read the file into a byte array so that we can parse the template
+	log.Printf("Reading template: %#v", args[0])
+	tpl, err := packer.ParseTemplateFile(args[0])
+	if err != nil {
+		env.Ui().Error(fmt.Sprintf("Failed to parse template: %s", err))
+		return 1
+	}
+
+	// Convenience...
+	ui := env.Ui()
+
+	// Variables
+	ui.Say("Variables and their defaults:\n")
+	if len(tpl.Variables) == 0 {
+		ui.Say("  <No variables>")
+	} else {
+		keys := make([]string, 0, len(tpl.Variables))
+		max := 0
+		for k, _ := range tpl.Variables {
+			keys = append(keys, k)
+			if len(k) > max {
+				max = len(k)
+			}
+		}
+
+		sort.Strings(keys)
+
+		for _, k := range keys {
+			v := tpl.Variables[k]
+			padding := strings.Repeat(" ", max-len(k))
+			output := fmt.Sprintf("  %s%s = %s", k, padding, v)
+
+			ui.Machine("template-variable", k, v)
+			ui.Say(output)
+		}
+	}
+
+	ui.Say("")
+
+	// Builders
+	ui.Say("Builders:\n")
+	if len(tpl.Builders) == 0 {
+		ui.Say("  <No builders>")
+	} else {
+		keys := make([]string, 0, len(tpl.Builders))
+		max := 0
+		for k, _ := range tpl.Builders {
+			keys = append(keys, k)
+			if len(k) > max {
+				max = len(k)
+			}
+		}
+
+		sort.Strings(keys)
+
+		for _, k := range keys {
+			v := tpl.Builders[k]
+			padding := strings.Repeat(" ", max-len(k))
+			output := fmt.Sprintf("  %s%s", k, padding)
+			if v.Name != v.Type {
+				output = fmt.Sprintf("%s (%s)", output, v.Type)
+			}
+
+			ui.Machine("template-builder", k, v.Type)
+			ui.Say(output)
+
+		}
+	}
+
+	ui.Say("")
+
+	// Provisioners
+	ui.Say("Provisioners:\n")
+	if len(tpl.Provisioners) == 0 {
+		ui.Say("  <No provisioners>")
+	} else {
+		for _, v := range tpl.Provisioners {
+			ui.Machine("template-provisioner", v.Type)
+			ui.Say(fmt.Sprintf("  %s", v.Type))
+		}
+	}
+
+	return 0
+}

command/inspect/command_test.go

@@ -0,0 +1,14 @@
+package inspect
+
+import (
+	"github.com/mitchellh/packer/packer"
+	"testing"
+)
+
+func TestCommand_Impl(t *testing.T) {
+	var raw interface{}
+	raw = new(Command)
+	if _, ok := raw.(packer.Command); !ok {
+		t.Fatalf("must be a Command")
+	}
+}

command/inspect/help.go

@@ -0,0 +1,13 @@
+package inspect
+
+const helpText = `
+Usage: packer inspect TEMPLATE
+
+  Inspects a template, parsing and outputting the components a template
+  defines. This does not validate the contents of a template (other than
+  basic syntax by necessity).
+
+Options:
+
+  -machine-readable  Machine-readable output
+`

common/config.go

@@ -64,6 +64,15 @@ func DecodeConfig(target interface{}, raws ...interface{}) (*mapstructure.Metada
 // a completely valid URL. For example, the original URL might be "local/file.iso"
 // which isn't a valid URL. DownloadableURL will return "file:///local/file.iso"
 func DownloadableURL(original string) (string, error) {
+	if runtime.GOOS == "windows" {
+		// If the distance to the first ":" is just one character, assume
+		// we're dealing with a drive letter and thus a file path.
+		idx := strings.Index(original, ":")
+		if idx == 1 {
+			original = "file:///" + original
+		}
+	}
+
 	url, err := url.Parse(original)
 	if err != nil {
 		return "", err

common/download.go

@@ -148,12 +148,12 @@ func (d *DownloadClient) Get() (string, error) {
 }
 
 // PercentProgress returns the download progress as a percentage.
-func (d *DownloadClient) PercentProgress() uint {
+func (d *DownloadClient) PercentProgress() int {
 	if d.downloader == nil {
-		return 0
+		return -1
 	}
 
-	return uint((float64(d.downloader.Progress()) / float64(d.downloader.Total())) * 100)
+	return int((float64(d.downloader.Progress()) / float64(d.downloader.Total())) * 100)
 }
 
 // VerifyChecksum tests that the path matches the checksum for the
@@ -188,7 +188,18 @@ func (*HTTPDownloader) Cancel() {
 
 func (d *HTTPDownloader) Download(dst io.Writer, src *url.URL) error {
 	log.Printf("Starting download: %s", src.String())
-	resp, err := http.Get(src.String())
+	req, err := http.NewRequest("GET", src.String(), nil)
+	if err != nil {
+		return err
+	}
+
+	httpClient := &http.Client{
+		Transport: &http.Transport{
+			Proxy: http.ProxyFromEnvironment,
+		},
+	}
+
+	resp, err := httpClient.Do(req)
 	if err != nil {
 		return err
 	}

common/step_download.go

@@ -0,0 +1,138 @@
+package common
+
+import (
+	"encoding/hex"
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/packer"
+	"log"
+	"time"
+)
+
+// StepDownload downloads a remote file using the download client within
+// this package. This step handles setting up the download configuration,
+// progress reporting, interrupt handling, etc.
+//
+// Uses:
+//   cache packer.Cache
+//   ui    packer.Ui
+type StepDownload struct {
+	// The checksum and the type of the checksum for the download
+	Checksum     string
+	ChecksumType string
+
+	// A short description of the type of download being done. Example:
+	// "ISO" or "Guest Additions"
+	Description string
+
+	// The name of the key where the final path of the ISO will be put
+	// into the state.
+	ResultKey string
+
+	// The path where the result should go, otherwise it goes to the
+	// cache directory.
+	TargetPath string
+
+	// A list of URLs to attempt to download this thing.
+	Url []string
+}
+
+func (s *StepDownload) Run(state map[string]interface{}) multistep.StepAction {
+	cache := state["cache"].(packer.Cache)
+	ui := state["ui"].(packer.Ui)
+
+	var checksum []byte
+	if s.Checksum != "" {
+		var err error
+		checksum, err = hex.DecodeString(s.Checksum)
+		if err != nil {
+			state["error"] = fmt.Errorf("Error parsing checksum: %s", err)
+			return multistep.ActionHalt
+		}
+	}
+
+	ui.Say(fmt.Sprintf("Downloading or copying %s", s.Description))
+
+	var finalPath string
+	for _, url := range s.Url {
+		ui.Message(fmt.Sprintf("Downloading or copying: %s", url))
+
+		targetPath := s.TargetPath
+		if targetPath == "" {
+			log.Printf("Acquiring lock to download: %s", url)
+			targetPath = cache.Lock(url)
+			defer cache.Unlock(url)
+		}
+
+		config := &DownloadConfig{
+			Url:        url,
+			TargetPath: targetPath,
+			CopyFile:   false,
+			Hash:       HashForType(s.ChecksumType),
+			Checksum:   checksum,
+		}
+
+		path, err, retry := s.download(config, state)
+		if err != nil {
+			ui.Message(fmt.Sprintf("Error downloading: %s", err))
+		}
+
+		if !retry {
+			return multistep.ActionHalt
+		}
+
+		if err == nil {
+			finalPath = path
+			break
+		}
+	}
+
+	if finalPath == "" {
+		err := fmt.Errorf("%s download failed.", s.Description)
+		state["error"] = err
+		ui.Error(err.Error())
+		return multistep.ActionHalt
+	}
+
+	state[s.ResultKey] = finalPath
+	return multistep.ActionContinue
+}
+
+func (s *StepDownload) Cleanup(map[string]interface{}) {}
+
+func (s *StepDownload) download(config *DownloadConfig, state map[string]interface{}) (string, error, bool) {
+	var path string
+	ui := state["ui"].(packer.Ui)
+	download := NewDownloadClient(config)
+
+	downloadCompleteCh := make(chan error, 1)
+	go func() {
+		var err error
+		path, err = download.Get()
+		downloadCompleteCh <- err
+	}()
+
+	progressTicker := time.NewTicker(5 * time.Second)
+	defer progressTicker.Stop()
+
+	for {
+		select {
+		case err := <-downloadCompleteCh:
+			if err != nil {
+				return "", err, true
+			}
+
+			return path, nil, true
+		case <-progressTicker.C:
+			progress := download.PercentProgress()
+			if progress >= 0 {
+				ui.Message(fmt.Sprintf("Download progress: %d%%", progress))
+			}
+		case <-time.After(1 * time.Second):
+			if _, ok := state[multistep.StateCancelled]; ok {
+				ui.Say("Interrupt received. Cancelling download...")
+				return "", nil, false
+			}
+		}
+	}
+}

common/step_download_test.go

@@ -0,0 +1,14 @@
+package common
+
+import (
+	"github.com/mitchellh/multistep"
+	"testing"
+)
+
+func TestStepDownload_Impl(t *testing.T) {
+	var raw interface{}
+	raw = new(StepDownload)
+	if _, ok := raw.(multistep.Step); !ok {
+		t.Fatalf("download should be a step")
+	}
+}

communicator/ssh/communicator.go

@@ -205,9 +205,14 @@ func (c *comm) Download(string, io.Writer) error {
 	panic("not implemented yet")
 }
 
-func (c *comm) newSession() (*ssh.Session, error) {
+func (c *comm) newSession() (session *ssh.Session, err error) {
 	log.Println("opening new ssh session")
-	session, err := c.client.NewSession()
+	if c.client == nil {
+		err = errors.New("client not available")
+	} else {
+		session, err = c.client.NewSession()
+	}
+
 	if err != nil {
 		log.Printf("ssh session open error: '%s', attempting reconnect", err)
 		if err := c.reconnect(); err != nil {
@@ -225,6 +230,10 @@ func (c *comm) reconnect() (err error) {
 		c.conn.Close()
 	}
 
+	// Set the conn and client to nil since we'll recreate it
+	c.conn = nil
+	c.client = nil
+
 	log.Printf("reconnecting to TCP connection for SSH")
 	c.conn, err = c.config.Connection()
 	if err != nil {

communicator/ssh/communicator_test.go

@@ -1,3 +1,5 @@
+// +build !race
+
 package ssh
 
 import (

config.go

@@ -30,6 +30,7 @@ const defaultConfig = `
 	"commands": {
 		"build": "packer-command-build",
 		"fix": "packer-command-fix",
+		"inspect": "packer-command-inspect",
 		"validate": "packer-command-validate"
 	},
 

packer.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"github.com/mitchellh/packer/packer"
 	"github.com/mitchellh/packer/packer/plugin"
+	"github.com/mitchellh/panicwrap"
 	"io"
 	"io/ioutil"
 	"log"
@@ -15,32 +16,68 @@ import (
 )
 
 func main() {
-	// Setup logging if PACKER_LOG is set.
-	// Log to PACKER_LOG_PATH if it is set, otherwise default to stderr.
-	var logOutput io.Writer = ioutil.Discard
-	if os.Getenv("PACKER_LOG") != "" {
-		logOutput = os.Stderr
-
-		if logPath := os.Getenv("PACKER_LOG_PATH"); logPath != "" {
-			var err error
-			logOutput, err = os.Create(logPath)
-			if err != nil {
-				fmt.Fprintf(
-					os.Stderr,
-					"Couldn't open '%s' for logging: %s",
-					logPath, err)
-				os.Exit(1)
-			}
-		}
-	}
-
-	log.SetOutput(logOutput)
+	// Call realMain instead of doing the work here so we can use
+	// `defer` statements within the function and have them work properly.
+	// (defers aren't called with os.Exit)
+	os.Exit(realMain())
+}
 
+// realMain is executed from main and returns the exit status to exit with.
+func realMain() int {
 	// If there is no explicit number of Go threads to use, then set it
 	if os.Getenv("GOMAXPROCS") == "" {
 		runtime.GOMAXPROCS(runtime.NumCPU())
 	}
 
+	// Determine where logs should go in general (requested by the user)
+	logWriter, err := logOutput()
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "Couldn't setup log output: %s", err)
+		return 1
+	}
+
+	// We also always send logs to a temporary file that we use in case
+	// there is a panic. Otherwise, we delete it.
+	logTempFile, err := ioutil.TempFile("", "packer-log")
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "Couldn't setup logging tempfile: %s", err)
+		return 1
+	}
+	defer os.Remove(logTempFile.Name())
+	defer logTempFile.Close()
+
+	// Reset the log variables to minimize work in the subprocess
+	os.Setenv("PACKER_LOG", "")
+	os.Setenv("PACKER_LOG_FILE", "")
+
+	// Create the configuration for panicwrap and wrap our executable
+	wrapConfig := &panicwrap.WrapConfig{
+		Handler: panicHandler(logTempFile),
+		Writer:  io.MultiWriter(logTempFile, logWriter),
+	}
+
+	exitStatus, err := panicwrap.Wrap(wrapConfig)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "Couldn't start Packer: %s", err)
+		return 1
+	}
+
+	if exitStatus >= 0 {
+		return exitStatus
+	}
+
+	// We're the child, so just close the tempfile we made in order to
+	// save file handles since the tempfile is only used by the parent.
+	logTempFile.Close()
+
+	return wrappedMain()
+}
+
+// wrappedMain is called only when we're wrapped by panicwrap and
+// returns the exit status to exit with.
+func wrappedMain() int {
+	log.SetOutput(os.Stderr)
+
 	log.Printf(
 		"Packer Version: %s %s %s",
 		packer.Version, packer.VersionPrerelease, packer.GitCommit)
@@ -52,7 +89,7 @@ func main() {
 	config, err := loadConfig()
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Error loading configuration: \n\n%s\n", err)
-		os.Exit(1)
+		return 1
 	}
 
 	log.Printf("Packer config: %+v", config)
@@ -65,12 +102,12 @@ func main() {
 	cacheDir, err = filepath.Abs(cacheDir)
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Error preparing cache directory: \n\n%s\n", err)
-		os.Exit(1)
+		return 1
 	}
 
 	if err := os.MkdirAll(cacheDir, 0755); err != nil {
 		fmt.Fprintf(os.Stderr, "Error preparing cache directory: \n\n%s\n", err)
-		os.Exit(1)
+		return 1
 	}
 
 	log.Printf("Setting cache directory: %s", cacheDir)
@@ -100,8 +137,7 @@ func main() {
 	env, err := packer.NewEnvironment(envConfig)
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Packer initialization error: \n\n%s\n", err)
-		plugin.CleanupClients()
-		os.Exit(1)
+		return 1
 	}
 
 	setupSignalHandlers(env)
@@ -109,12 +145,10 @@ func main() {
 	exitCode, err := env.Cli(args)
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Error executing CLI: %s\n", err.Error())
-		plugin.CleanupClients()
-		os.Exit(1)
+		return 1
 	}
 
-	plugin.CleanupClients()
-	os.Exit(exitCode)
+	return exitCode
 }
 
 // extractMachineReadable checks the args for the machine readable
@@ -178,3 +212,21 @@ func loadConfig() (*config, error) {
 
 	return &config, nil
 }
+
+// logOutput determines where we should send logs (if anywhere).
+func logOutput() (logOutput io.Writer, err error) {
+	logOutput = ioutil.Discard
+	if os.Getenv("PACKER_LOG") != "" {
+		logOutput = os.Stderr
+
+		if logPath := os.Getenv("PACKER_LOG_PATH"); logPath != "" {
+			var err error
+			logOutput, err = os.Create(logPath)
+			if err != nil {
+				return nil, err
+			}
+		}
+	}
+
+	return
+}

packer/build.go

@@ -90,7 +90,7 @@ type coreBuild struct {
 type coreBuildPostProcessor struct {
 	processor         PostProcessor
 	processorType     string
-	config            interface{}
+	config            map[string]interface{}
 	keepInputArtifact bool
 }
 

packer/build_test.go

@@ -20,7 +20,7 @@ func testBuild() *coreBuild {
 		},
 		postProcessors: [][]coreBuildPostProcessor{
 			[]coreBuildPostProcessor{
-				coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp"}, "testPP", 42, true},
+				coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp"}, "testPP", make(map[string]interface{}), true},
 			},
 		},
 		variables: make(map[string]string),
@@ -66,7 +66,7 @@ func TestBuild_Prepare(t *testing.T) {
 	corePP := build.postProcessors[0][0]
 	pp := corePP.processor.(*TestPostProcessor)
 	assert.True(pp.configCalled, "config should be called")
-	assert.Equal(pp.configVal, []interface{}{42, packerConfig}, "config should have right value")
+	assert.Equal(pp.configVal, []interface{}{make(map[string]interface{}), packerConfig}, "config should have right value")
 }
 
 func TestBuild_Prepare_Twice(t *testing.T) {
@@ -231,7 +231,7 @@ func TestBuild_Run_Artifacts(t *testing.T) {
 	build = testBuild()
 	build.postProcessors = [][]coreBuildPostProcessor{
 		[]coreBuildPostProcessor{
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp"}, "pp", 42, false},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp"}, "pp", make(map[string]interface{}), false},
 		},
 	}
 
@@ -256,10 +256,10 @@ func TestBuild_Run_Artifacts(t *testing.T) {
 	build = testBuild()
 	build.postProcessors = [][]coreBuildPostProcessor{
 		[]coreBuildPostProcessor{
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp1"}, "pp", 42, false},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp1"}, "pp", make(map[string]interface{}), false},
 		},
 		[]coreBuildPostProcessor{
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp2"}, "pp", 42, true},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp2"}, "pp", make(map[string]interface{}), true},
 		},
 	}
 
@@ -284,12 +284,12 @@ func TestBuild_Run_Artifacts(t *testing.T) {
 	build = testBuild()
 	build.postProcessors = [][]coreBuildPostProcessor{
 		[]coreBuildPostProcessor{
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp1a"}, "pp", 42, false},
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp1b"}, "pp", 42, true},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp1a"}, "pp", make(map[string]interface{}), false},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp1b"}, "pp", make(map[string]interface{}), true},
 		},
 		[]coreBuildPostProcessor{
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp2a"}, "pp", 42, false},
-			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp2b"}, "pp", 42, false},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp2a"}, "pp", make(map[string]interface{}), false},
+			coreBuildPostProcessor{&TestPostProcessor{artifactId: "pp2b"}, "pp", make(map[string]interface{}), false},
 		},
 	}
 
@@ -315,7 +315,7 @@ func TestBuild_Run_Artifacts(t *testing.T) {
 	build.postProcessors = [][]coreBuildPostProcessor{
 		[]coreBuildPostProcessor{
 			coreBuildPostProcessor{
-				&TestPostProcessor{artifactId: "pp", keep: true}, "pp", 42, false,
+				&TestPostProcessor{artifactId: "pp", keep: true}, "pp", make(map[string]interface{}), false,
 			},
 		},
 	}

packer/communicator.go

@@ -33,9 +33,11 @@ type RemoteCmd struct {
 	// Once Exited is true, this will contain the exit code of the process.
 	ExitStatus int
 
-	// Internal locks and such used for safely setting some shared variables
-	l      sync.Mutex
+	// Internal fields
 	exitCh chan struct{}
+
+	// This thing is a mutex, lock when making modifications concurrently
+	sync.Mutex
 }
 
 // A Communicator is the interface used to communicate with the machine
@@ -72,6 +74,17 @@ func (r *RemoteCmd) StartWithUi(c Communicator, ui Ui) error {
 	defer stdout_w.Close()
 	defer stderr_w.Close()
 
+	// Retain the original stdout/stderr that we can replace back in.
+	originalStdout := r.Stdout
+	originalStderr := r.Stderr
+	defer func() {
+		r.Lock()
+		defer r.Unlock()
+
+		r.Stdout = originalStdout
+		r.Stderr = originalStderr
+	}()
+
 	// Set the writers for the output so that we get it streamed to us
 	if r.Stdout == nil {
 		r.Stdout = stdout_w
@@ -108,9 +121,9 @@ OutputLoop:
 	for {
 		select {
 		case output := <-stderrCh:
-			ui.Message(strings.TrimSpace(output))
+			ui.Message(r.cleanOutputLine(output))
 		case output := <-stdoutCh:
-			ui.Message(strings.TrimSpace(output))
+			ui.Message(r.cleanOutputLine(output))
 		case <-exitCh:
 			break OutputLoop
 		}
@@ -133,8 +146,8 @@ OutputLoop:
 // should be called by communicators who are running a remote command in
 // order to set that the command is done.
 func (r *RemoteCmd) SetExited(status int) {
-	r.l.Lock()
-	defer r.l.Unlock()
+	r.Lock()
+	defer r.Unlock()
 
 	if r.exitCh == nil {
 		r.exitCh = make(chan struct{})
@@ -148,11 +161,27 @@ func (r *RemoteCmd) SetExited(status int) {
 // Wait waits for the remote command to complete.
 func (r *RemoteCmd) Wait() {
 	// Make sure our condition variable is initialized.
-	r.l.Lock()
+	r.Lock()
 	if r.exitCh == nil {
 		r.exitCh = make(chan struct{})
 	}
-	r.l.Unlock()
+	r.Unlock()
 
 	<-r.exitCh
 }
+
+// cleanOutputLine cleans up a line so that '\r' don't muck up the
+// UI output when we're reading from a remote command.
+func (r *RemoteCmd) cleanOutputLine(line string) string {
+	// Trim surrounding whitespace
+	line = strings.TrimSpace(line)
+
+	// Trim up to the first carriage return, since that text would be
+	// lost anyways.
+	idx := strings.LastIndex(line, "\r")
+	if idx > -1 {
+		line = line[idx+1:]
+	}
+
+	return line
+}

packer/communicator_test.go

@@ -15,6 +15,9 @@ type TestCommunicator struct {
 
 func (c *TestCommunicator) Start(rc *RemoteCmd) error {
 	go func() {
+		rc.Lock()
+		defer rc.Unlock()
+
 		if rc.Stdout != nil && c.Stdout != nil {
 			io.Copy(rc.Stdout, c.Stdout)
 		}

packer/config_template.go

@@ -1,4 +1,4 @@
-package common
+package packer
 
 import (
 	"bytes"
@@ -8,19 +8,19 @@ import (
 	"time"
 )
 
-// Template processes string data as a text/template with some common
+// ConfigTemplate processes string data as a text/template with some common
 // elements and functions available. Plugin creators should process as
 // many fields as possible through this.
-type Template struct {
+type ConfigTemplate struct {
 	UserVars map[string]string
 
 	root *template.Template
 	i    int
 }
 
-// NewTemplate creates a new template processor.
-func NewTemplate() (*Template, error) {
-	result := &Template{
+// NewConfigTemplate creates a new configuration template processor.
+func NewConfigTemplate() (*ConfigTemplate, error) {
+	result := &ConfigTemplate{
 		UserVars: make(map[string]string),
 	}
 
@@ -34,7 +34,7 @@ func NewTemplate() (*Template, error) {
 }
 
 // Process processes a single string, compiling and executing the template.
-func (t *Template) Process(s string, data interface{}) (string, error) {
+func (t *ConfigTemplate) Process(s string, data interface{}) (string, error) {
 	tpl, err := t.root.New(t.nextTemplateName()).Parse(s)
 	if err != nil {
 		return "", err
@@ -49,7 +49,7 @@ func (t *Template) Process(s string, data interface{}) (string, error) {
 }
 
 // Validate the template.
-func (t *Template) Validate(s string) error {
+func (t *ConfigTemplate) Validate(s string) error {
 	root, err := t.root.Clone()
 	if err != nil {
 		return err
@@ -59,7 +59,7 @@ func (t *Template) Validate(s string) error {
 	return err
 }
 
-func (t *Template) nextTemplateName() string {
+func (t *ConfigTemplate) nextTemplateName() string {
 	name := fmt.Sprintf("tpl%d", t.i)
 	t.i++
 	return name
@@ -67,7 +67,7 @@ func (t *Template) nextTemplateName() string {
 
 // User is the function exposed as "user" within the templates and
 // looks up user variables.
-func (t *Template) templateUser(n string) (string, error) {
+func (t *ConfigTemplate) templateUser(n string) (string, error) {
 	result, ok := t.UserVars[n]
 	if !ok {
 		return "", fmt.Errorf("uknown user var: %s", n)

packer/config_template_test.go

@@ -1,4 +1,4 @@
-package common
+package packer
 
 import (
 	"math"
@@ -7,8 +7,8 @@ import (
 	"time"
 )
 
-func TestTemplateProcess_timestamp(t *testing.T) {
-	tpl, err := NewTemplate()
+func TestConfigTemplateProcess_timestamp(t *testing.T) {
+	tpl, err := NewConfigTemplate()
 	if err != nil {
 		t.Fatalf("err: %s", err)
 	}
@@ -29,8 +29,8 @@ func TestTemplateProcess_timestamp(t *testing.T) {
 	}
 }
 
-func TestTemplateProcess_user(t *testing.T) {
-	tpl, err := NewTemplate()
+func TestConfigTemplateProcess_user(t *testing.T) {
+	tpl, err := NewConfigTemplate()
 	if err != nil {
 		t.Fatalf("err: %s", err)
 	}
@@ -47,8 +47,8 @@ func TestTemplateProcess_user(t *testing.T) {
 	}
 }
 
-func TestTemplateValidate(t *testing.T) {
-	tpl, err := NewTemplate()
+func TestConfigTemplateValidate(t *testing.T) {
+	tpl, err := NewConfigTemplate()
 	if err != nil {
 		t.Fatalf("err: %s", err)
 	}

packer/environment.go

@@ -8,6 +8,7 @@ import (
 	"os"
 	"sort"
 	"strings"
+	"sync"
 )
 
 // The function type used to lookup Builder implementations.
@@ -68,6 +69,12 @@ type EnvironmentConfig struct {
 	Ui         Ui
 }
 
+type helpCommandEntry struct {
+	i        int
+	key      string
+	synopsis string
+}
+
 // DefaultEnvironmentConfig returns a default EnvironmentConfig that can
 // be used to create a new enviroment with NewEnvironment with sane defaults.
 func DefaultEnvironmentConfig() *EnvironmentConfig {
@@ -245,7 +252,7 @@ func (e *coreEnvironment) Cli(args []string) (result int, err error) {
 
 		// If we still don't have a command, show the help.
 		if command == nil {
-			log.Printf("Environment.CLI: command not found: %s\n", args[0])
+			e.ui.Error(fmt.Sprintf("Unknown command: %s\n", args[0]))
 			e.printHelp()
 			return 1, nil
 		}
@@ -266,42 +273,79 @@ func (e *coreEnvironment) Cli(args []string) (result int, err error) {
 func (e *coreEnvironment) printHelp() {
 	// Created a sorted slice of the map keys and record the longest
 	// command name so we can better format the output later.
-	i := 0
 	maxKeyLen := 0
 	for _, command := range e.commands {
 		if len(command) > maxKeyLen {
 			maxKeyLen = len(command)
 		}
-
-		i++
 	}
 
 	// Sort the keys
 	sort.Strings(e.commands)
 
+	// Create the communication/sync mechanisms to get the synopsis' of
+	// the various commands. We do this in parallel since the overhead
+	// of the subprocess underneath is very expensive and this speeds things
+	// up an incredible amount.
+	var wg sync.WaitGroup
+	ch := make(chan *helpCommandEntry)
+
+	for i, key := range e.commands {
+		wg.Add(1)
+
+		// Get the synopsis in a goroutine since it may take awhile
+		// to subprocess out.
+		go func(i int, key string) {
+			defer wg.Done()
+			var synopsis string
+			command, err := e.components.Command(key)
+			if err != nil {
+				synopsis = fmt.Sprintf("Error loading command: %s", err.Error())
+			} else if command == nil {
+				return
+			} else {
+				synopsis = command.Synopsis()
+			}
+
+			// Pad the key with spaces so that they're all the same width
+			key = fmt.Sprintf("%s%s", key, strings.Repeat(" ", maxKeyLen-len(key)))
+
+			// Output the command and the synopsis
+			ch <- &helpCommandEntry{
+				i:        i,
+				key:      key,
+				synopsis: synopsis,
+			}
+		}(i, key)
+	}
+
 	e.ui.Say("usage: packer [--version] [--help] <command> [<args>]\n")
 	e.ui.Say("Available commands are:")
-	for _, key := range e.commands {
-		var synopsis string
 
-		command, err := e.components.Command(key)
-		if err != nil {
-			synopsis = fmt.Sprintf("Error loading command: %s", err.Error())
-		} else if command == nil {
-			continue
-		} else {
-			synopsis = command.Synopsis()
+	// Make a goroutine that just waits for all the synopsis gathering
+	// to complete, and then output it.
+	synopsisDone := make(chan struct{})
+	go func() {
+		defer close(synopsisDone)
+		entries := make([]string, len(e.commands))
+
+		for entry := range ch {
+			e.ui.Machine("command", entry.key, entry.synopsis)
+			message := fmt.Sprintf("    %s    %s", entry.key, entry.synopsis)
+			entries[entry.i] = message
 		}
 
-		// Machine-readable output of the available command
-		e.ui.Machine("command", key, synopsis)
+		for _, message := range entries {
+			if message != "" {
+				e.ui.Say(message)
+			}
+		}
+	}()
 
-		// Pad the key with spaces so that they're all the same width
-		key = fmt.Sprintf("%v%v", key, strings.Repeat(" ", maxKeyLen-len(key)))
-
-		// Output the command and the synopsis
-		e.ui.Say(fmt.Sprintf("    %v     %v", key, synopsis))
-	}
+	// Wait to complete getting the synopsis' then close the channel
+	wg.Wait()
+	close(ch)
+	<-synopsisDone
 
 	e.ui.Say("\nGlobally recognized options:")
 	e.ui.Say("    -machine-readable    Machine-readable output format.")

packer/environment_test.go

@@ -17,6 +17,17 @@ func init() {
 	log.SetOutput(ioutil.Discard)
 }
 
+func testComponentFinder() *ComponentFinder {
+	builderFactory := func(n string) (Builder, error) { return testBuilder(), nil }
+	ppFactory := func(n string) (PostProcessor, error) { return new(TestPostProcessor), nil }
+	provFactory := func(n string) (Provisioner, error) { return new(TestProvisioner), nil }
+	return &ComponentFinder{
+		Builder:       builderFactory,
+		PostProcessor: ppFactory,
+		Provisioner:   provFactory,
+	}
+}
+
 func testEnvironment() Environment {
 	config := DefaultEnvironmentConfig()
 	config.Ui = &BasicUi{

packer/plugin/builder.go

@@ -40,7 +40,7 @@ func (b *cmdBuilder) Cancel() {
 func (c *cmdBuilder) checkExit(p interface{}, cb func()) {
 	if c.client.Exited() && cb != nil {
 		cb()
-	} else if p != nil {
+	} else if p != nil && !Killed {
 		log.Panic(p)
 	}
 }

packer/plugin/client.go

@@ -2,7 +2,6 @@ package plugin
 
 import (
 	"bufio"
-	"bytes"
 	"errors"
 	"fmt"
 	"github.com/mitchellh/packer/packer"
@@ -10,6 +9,7 @@ import (
 	"io"
 	"io/ioutil"
 	"log"
+	"net"
 	"net/rpc"
 	"os"
 	"os/exec"
@@ -19,6 +19,10 @@ import (
 	"unicode"
 )
 
+// If this is true, then the "unexpected EOF" panic will not be
+// raised throughout the clients.
+var Killed = false
+
 // This is a slice of the "managed" clients which are cleaned up when
 // calling Cleanup
 var managedClients = make([]*Client, 0, 5)
@@ -68,6 +72,9 @@ type ClientConfig struct {
 //
 // This must only be called _once_.
 func CleanupClients() {
+	// Set the killed to true so that we don't get unexpected panics
+	Killed = true
+
 	// Kill all the managed clients in parallel and use a WaitGroup
 	// to wait for them all to finish up.
 	var wg sync.WaitGroup
@@ -115,6 +122,8 @@ func NewClient(config *ClientConfig) (c *Client) {
 
 // Tells whether or not the underlying process has exited.
 func (c *Client) Exited() bool {
+	c.l.Lock()
+	defer c.l.Unlock()
 	return c.exited
 }
 
@@ -214,7 +223,7 @@ func (c *Client) Start() (address string, err error) {
 		fmt.Sprintf("PACKER_PLUGIN_MAX_PORT=%d", c.config.MaxPort),
 	}
 
-	stdout := new(bytes.Buffer)
+	stdout_r, stdout_w := io.Pipe()
 	stderr_r, stderr_w := io.Pipe()
 
 	cmd := c.config.Cmd
@@ -222,7 +231,7 @@ func (c *Client) Start() (address string, err error) {
 	cmd.Env = append(cmd.Env, env...)
 	cmd.Stdin = os.Stdin
 	cmd.Stderr = stderr_w
-	cmd.Stdout = stdout
+	cmd.Stdout = stdout_w
 
 	log.Printf("Starting plugin: %s %#v", cmd.Path, cmd.Args)
 	err = cmd.Start()
@@ -244,10 +253,12 @@ func (c *Client) Start() (address string, err error) {
 	}()
 
 	// Start goroutine to wait for process to exit
+	exitCh := make(chan struct{})
 	go func() {
-		// Make sure we close the write end of our stderr listener so
-		// that the log goroutine ends properly.
+		// Make sure we close the write end of our stderr/stdout so
+		// that the readers send EOF properly.
 		defer stderr_w.Close()
+		defer stdout_w.Close()
 
 		// Wait for the command to end.
 		cmd.Wait()
@@ -257,46 +268,60 @@ func (c *Client) Start() (address string, err error) {
 		os.Stderr.Sync()
 
 		// Mark that we exited
+		close(exitCh)
+
+		// Set that we exited, which takes a lock
+		c.l.Lock()
+		defer c.l.Unlock()
 		c.exited = true
 	}()
 
 	// Start goroutine that logs the stderr
 	go c.logStderr(stderr_r)
 
+	// Start a goroutine that is going to be reading the lines
+	// out of stdout
+	linesCh := make(chan []byte)
+	go func() {
+		defer close(linesCh)
+
+		buf := bufio.NewReader(stdout_r)
+		for {
+			line, err := buf.ReadBytes('\n')
+			if line != nil {
+				linesCh <- line
+			}
+
+			if err == io.EOF {
+				return
+			}
+		}
+	}()
+
+	// Make sure after we exit we read the lines from stdout forever
+	// so they dont' block since it is an io.Pipe
+	defer func() {
+		go func() {
+			for _ = range linesCh {
+			}
+		}()
+	}()
+
 	// Some channels for the next step
 	timeout := time.After(c.config.StartTimeout)
 
 	// Start looking for the address
 	log.Printf("Waiting for RPC address for: %s", cmd.Path)
-	for done := false; !done; {
-		select {
-		case <-timeout:
-			err = errors.New("timeout while waiting for plugin to start")
-			done = true
-		default:
-		}
-
-		if err == nil && c.Exited() {
-			err = errors.New("plugin exited before we could connect")
-			done = true
-		}
-
-		if line, lerr := stdout.ReadBytes('\n'); lerr == nil {
-			// Trim the address and reset the err since we were able
-			// to read some sort of address.
-			c.address = strings.TrimSpace(string(line))
-			address = c.address
-			err = nil
-			break
-		}
-
-		// If error is nil from previously, return now
-		if err != nil {
-			return
-		}
-
-		// Wait a bit
-		time.Sleep(10 * time.Millisecond)
+	select {
+	case <-timeout:
+		err = errors.New("timeout while waiting for plugin to start")
+	case <-exitCh:
+		err = errors.New("plugin exited before we could connect")
+	case line := <-linesCh:
+		// Trim the address and reset the err since we were able
+		// to read some sort of address.
+		c.address = strings.TrimSpace(string(line))
+		address = c.address
 	}
 
 	return
@@ -328,10 +353,14 @@ func (c *Client) rpcClient() (*rpc.Client, error) {
 		return nil, err
 	}
 
-	client, err := rpc.Dial("tcp", address)
+	conn, err := net.Dial("tcp", address)
 	if err != nil {
 		return nil, err
 	}
 
-	return client, nil
+	// Make sure to set keep alive so that the connection doesn't die
+	tcpConn := conn.(*net.TCPConn)
+	tcpConn.SetKeepAlive(true)
+
+	return rpc.NewClient(tcpConn), nil
 }

packer/plugin/command.go

@@ -45,7 +45,7 @@ func (c *cmdCommand) Synopsis() (result string) {
 func (c *cmdCommand) checkExit(p interface{}, cb func()) {
 	if c.client.Exited() {
 		cb()
-	} else if p != nil {
+	} else if p != nil && !Killed {
 		log.Panic(p)
 	}
 }

packer/plugin/hook.go

@@ -22,7 +22,7 @@ func (c *cmdHook) Run(name string, ui packer.Ui, comm packer.Communicator, data
 func (c *cmdHook) checkExit(p interface{}, cb func()) {
 	if c.client.Exited() {
 		cb()
-	} else if p != nil {
+	} else if p != nil && !Killed {
 		log.Panic(p)
 	}
 }

packer/plugin/post_processor.go

@@ -31,7 +31,7 @@ func (c *cmdPostProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.
 func (c *cmdPostProcessor) checkExit(p interface{}, cb func()) {
 	if c.client.Exited() {
 		cb()
-	} else if p != nil {
+	} else if p != nil && !Killed {
 		log.Panic(p)
 	}
 }

packer/plugin/provisioner.go

@@ -31,7 +31,7 @@ func (c *cmdProvisioner) Provision(ui packer.Ui, comm packer.Communicator) error
 func (c *cmdProvisioner) checkExit(p interface{}, cb func()) {
 	if c.client.Exited() {
 		cb()
-	} else if p != nil {
+	} else if p != nil && !Killed {
 		log.Panic(p)
 	}
 }

packer/rpc/communicator.go

@@ -78,7 +78,8 @@ func (c *communicator) Start(cmd *packer.RemoteCmd) (err error) {
 
 		conn, err := responseL.Accept()
 		if err != nil {
-			log.Panic(err)
+			cmd.SetExited(123)
+			return
 		}
 
 		defer conn.Close()
@@ -87,7 +88,8 @@ func (c *communicator) Start(cmd *packer.RemoteCmd) (err error) {
 
 		var finished CommandFinished
 		if err := decoder.Decode(&finished); err != nil {
-			log.Panic(err)
+			cmd.SetExited(123)
+			return
 		}
 
 		cmd.SetExited(finished.ExitStatus)

packer/rpc/communicator_test.go

@@ -99,7 +99,10 @@ func TestCommunicatorRPC(t *testing.T) {
 	c.startCmd.SetExited(42)
 
 	for i := 0; i < 5; i++ {
-		if cmd.Exited {
+		cmd.Lock()
+		exited := cmd.Exited
+		cmd.Unlock()
+		if exited {
 			assert.Equal(cmd.ExitStatus, 42, "should have proper exit status")
 			break
 		}

packer/template.go

@@ -1,10 +1,13 @@
 package packer
 
 import (
+	"bytes"
 	"fmt"
 	"github.com/mitchellh/mapstructure"
 	jsonutil "github.com/mitchellh/packer/common/json"
+	"io"
 	"io/ioutil"
+	"os"
 	"sort"
 )
 
@@ -24,40 +27,40 @@ type rawTemplate struct {
 // completed form it can be without additional processing by the caller.
 type Template struct {
 	Variables      map[string]string
-	Builders       map[string]rawBuilderConfig
+	Builders       map[string]RawBuilderConfig
 	Hooks          map[string][]string
-	PostProcessors [][]rawPostProcessorConfig
-	Provisioners   []rawProvisionerConfig
+	PostProcessors [][]RawPostProcessorConfig
+	Provisioners   []RawProvisionerConfig
 }
 
-// The rawBuilderConfig struct represents a raw, unprocessed builder
+// The RawBuilderConfig struct represents a raw, unprocessed builder
 // configuration. It contains the name of the builder as well as the
 // raw configuration. If requested, this is used to compile into a full
 // builder configuration at some point.
-type rawBuilderConfig struct {
+type RawBuilderConfig struct {
 	Name string
 	Type string
 
-	rawConfig interface{}
+	RawConfig interface{}
 }
 
-// rawPostProcessorConfig represents a raw, unprocessed post-processor
+// RawPostProcessorConfig represents a raw, unprocessed post-processor
 // configuration. It contains the type of the post processor as well as the
 // raw configuration that is handed to the post-processor for it to process.
-type rawPostProcessorConfig struct {
+type RawPostProcessorConfig struct {
 	Type              string
 	KeepInputArtifact bool `mapstructure:"keep_input_artifact"`
-	rawConfig         interface{}
+	RawConfig         map[string]interface{}
 }
 
-// rawProvisionerConfig represents a raw, unprocessed provisioner configuration.
+// RawProvisionerConfig represents a raw, unprocessed provisioner configuration.
 // It contains the type of the provisioner as well as the raw configuration
 // that is handed to the provisioner for it to process.
-type rawProvisionerConfig struct {
+type RawProvisionerConfig struct {
 	Type     string
 	Override map[string]interface{}
 
-	rawConfig interface{}
+	RawConfig interface{}
 }
 
 // ParseTemplate takes a byte slice and parses a Template from it, returning
@@ -103,10 +106,10 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 
 	t = &Template{}
 	t.Variables = make(map[string]string)
-	t.Builders = make(map[string]rawBuilderConfig)
+	t.Builders = make(map[string]RawBuilderConfig)
 	t.Hooks = rawTpl.Hooks
-	t.PostProcessors = make([][]rawPostProcessorConfig, len(rawTpl.PostProcessors))
-	t.Provisioners = make([]rawProvisionerConfig, len(rawTpl.Provisioners))
+	t.PostProcessors = make([][]RawPostProcessorConfig, len(rawTpl.PostProcessors))
+	t.Provisioners = make([]RawProvisionerConfig, len(rawTpl.Provisioners))
 
 	// Gather all the variables
 	for k, v := range rawTpl.Variables {
@@ -115,7 +118,7 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 
 	// Gather all the builders
 	for i, v := range rawTpl.Builders {
-		var raw rawBuilderConfig
+		var raw RawBuilderConfig
 		if err := mapstructure.Decode(v, &raw); err != nil {
 			if merr, ok := err.(*mapstructure.Error); ok {
 				for _, err := range merr.Errors {
@@ -150,7 +153,7 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 		// itself doesn't know about, and it will cause a validation error.
 		delete(v, "name")
 
-		raw.rawConfig = v
+		raw.RawConfig = v
 
 		t.Builders[raw.Name] = raw
 	}
@@ -159,13 +162,13 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 	// are actually three different formats that the user can use to define
 	// a post-processor.
 	for i, rawV := range rawTpl.PostProcessors {
-		rawPP, err := parsePostProvisioner(i, rawV)
+		rawPP, err := parsePostProcessor(i, rawV)
 		if err != nil {
 			errors = append(errors, err...)
 			continue
 		}
 
-		t.PostProcessors[i] = make([]rawPostProcessorConfig, len(rawPP))
+		t.PostProcessors[i] = make([]RawPostProcessorConfig, len(rawPP))
 		configs := t.PostProcessors[i]
 		for j, pp := range rawPP {
 			config := &configs[j]
@@ -186,7 +189,10 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 				continue
 			}
 
-			config.rawConfig = pp
+			// Remove the input keep_input_artifact option
+			delete(pp, "keep_input_artifact")
+
+			config.RawConfig = pp
 		}
 	}
 
@@ -215,7 +221,7 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 		// actively reject them as invalid configuration.
 		delete(v, "override")
 
-		raw.rawConfig = v
+		raw.RawConfig = v
 	}
 
 	if len(t.Builders) == 0 {
@@ -235,15 +241,29 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 // ParseTemplateFile takes the given template file and parses it into
 // a single template.
 func ParseTemplateFile(path string) (*Template, error) {
-	data, err := ioutil.ReadFile(path)
-	if err != nil {
-		return nil, err
+	var data []byte
+
+	if path == "-" {
+		// Read from stdin...
+		buf := new(bytes.Buffer)
+		_, err := io.Copy(buf, os.Stdin)
+		if err != nil {
+			return nil, err
+		}
+
+		data = buf.Bytes()
+	} else {
+		var err error
+		data, err = ioutil.ReadFile(path)
+		if err != nil {
+			return nil, err
+		}
 	}
 
 	return ParseTemplate(data)
 }
 
-func parsePostProvisioner(i int, rawV interface{}) (result []map[string]interface{}, errors []error) {
+func parsePostProcessor(i int, rawV interface{}) (result []map[string]interface{}, errors []error) {
 	switch v := rawV.(type) {
 	case string:
 		result = []map[string]interface{}{
@@ -365,7 +385,7 @@ func (t *Template) Build(name string, components *ComponentFinder) (b Build, err
 			current[i] = coreBuildPostProcessor{
 				processor:         pp,
 				processorType:     rawPP.Type,
-				config:            rawPP.rawConfig,
+				config:            rawPP.RawConfig,
 				keepInputArtifact: rawPP.KeepInputArtifact,
 			}
 		}
@@ -388,7 +408,7 @@ func (t *Template) Build(name string, components *ComponentFinder) (b Build, err
 		}
 
 		configs := make([]interface{}, 1, 2)
-		configs[0] = rawProvisioner.rawConfig
+		configs[0] = rawProvisioner.RawConfig
 
 		if rawProvisioner.Override != nil {
 			if override, ok := rawProvisioner.Override[name]; ok {
@@ -403,7 +423,7 @@ func (t *Template) Build(name string, components *ComponentFinder) (b Build, err
 	b = &coreBuild{
 		name:           name,
 		builder:        builder,
-		builderConfig:  builderConfig.rawConfig,
+		builderConfig:  builderConfig.RawConfig,
 		builderType:    builderConfig.Type,
 		hooks:          hooks,
 		postProcessors: postProcessors,

packer/template_test.go

@@ -3,22 +3,12 @@ package packer
 import (
 	"cgl.tideland.biz/asserts"
 	"io/ioutil"
+	"os"
 	"reflect"
 	"sort"
 	"testing"
 )
 
-func testComponentFinder() *ComponentFinder {
-	builderFactory := func(n string) (Builder, error) { return testBuilder(), nil }
-	ppFactory := func(n string) (PostProcessor, error) { return new(TestPostProcessor), nil }
-	provFactory := func(n string) (Provisioner, error) { return new(TestProvisioner), nil }
-	return &ComponentFinder{
-		Builder:       builderFactory,
-		PostProcessor: ppFactory,
-		Provisioner:   provFactory,
-	}
-}
-
 func TestParseTemplateFile_basic(t *testing.T) {
 	data := `
 	{
@@ -43,6 +33,39 @@ func TestParseTemplateFile_basic(t *testing.T) {
 	}
 }
 
+func TestParseTemplateFile_stdin(t *testing.T) {
+	data := `
+	{
+		"builders": [{"type": "something"}]
+	}
+	`
+
+	tf, err := ioutil.TempFile("", "packer")
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+	defer tf.Close()
+	tf.Write([]byte(data))
+
+	// Sync and seek to the beginning so that we can re-read the contents
+	tf.Sync()
+	tf.Seek(0, 0)
+
+	// Set stdin to something we control
+	oldStdin := os.Stdin
+	defer func() { os.Stdin = oldStdin }()
+	os.Stdin = tf
+
+	result, err := ParseTemplateFile("-")
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+
+	if len(result.Builders) != 1 {
+		t.Fatalf("bad: %#v", result.Builders)
+	}
+}
+
 func TestParseTemplate_Basic(t *testing.T) {
 	assert := asserts.NewTestingAsserts(t, true)
 
@@ -165,8 +188,8 @@ func TestParseTemplate_BuilderWithName(t *testing.T) {
 	assert.True(ok, "should have bob builder")
 	assert.Equal(builder.Type, "amazon-ebs", "builder should be amazon-ebs")
 
-	rawConfig := builder.rawConfig
-	if rawConfig == nil {
+	RawConfig := builder.RawConfig
+	if RawConfig == nil {
 		t.Fatal("missing builder raw config")
 	}
 
@@ -174,8 +197,8 @@ func TestParseTemplate_BuilderWithName(t *testing.T) {
 		"type": "amazon-ebs",
 	}
 
-	if !reflect.DeepEqual(rawConfig, expected) {
-		t.Fatalf("bad raw: %#v", rawConfig)
+	if !reflect.DeepEqual(RawConfig, expected) {
+		t.Fatalf("bad raw: %#v", RawConfig)
 	}
 }
 
@@ -333,7 +356,7 @@ func TestParseTemplate_Provisioners(t *testing.T) {
 	assert.NotNil(result, "template should not be nil")
 	assert.Length(result.Provisioners, 1, "should have one provisioner")
 	assert.Equal(result.Provisioners[0].Type, "shell", "provisioner should be shell")
-	assert.NotNil(result.Provisioners[0].rawConfig, "should have raw config")
+	assert.NotNil(result.Provisioners[0].RawConfig, "should have raw config")
 }
 
 func TestParseTemplate_Variables(t *testing.T) {
@@ -600,6 +623,11 @@ func TestTemplate_Build(t *testing.T) {
 	assert.Equal(len(coreBuild.postProcessors[1]), 2, "should have correct number")
 	assert.False(coreBuild.postProcessors[1][0].keepInputArtifact, "shoule be correct")
 	assert.True(coreBuild.postProcessors[1][1].keepInputArtifact, "shoule be correct")
+
+	config := coreBuild.postProcessors[1][1].config
+	if _, ok := config["keep_input_artifact"]; ok {
+		t.Fatal("should not have keep_input_artifact")
+	}
 }
 
 func TestTemplate_Build_ProvisionerOverride(t *testing.T) {
@@ -631,8 +659,8 @@ func TestTemplate_Build_ProvisionerOverride(t *testing.T) {
 		t.Fatalf("err: %s", err)
 	}
 
-	rawConfig := template.Provisioners[0].rawConfig
-	if rawConfig == nil {
+	RawConfig := template.Provisioners[0].RawConfig
+	if RawConfig == nil {
 		t.Fatal("missing provisioner raw config")
 	}
 
@@ -640,8 +668,8 @@ func TestTemplate_Build_ProvisionerOverride(t *testing.T) {
 		"type": "test-prov",
 	}
 
-	if !reflect.DeepEqual(rawConfig, expected) {
-		t.Fatalf("bad raw: %#v", rawConfig)
+	if !reflect.DeepEqual(RawConfig, expected) {
+		t.Fatalf("bad raw: %#v", RawConfig)
 	}
 
 	builder := testBuilder()

packer/version.go

@@ -10,7 +10,7 @@ import (
 var GitCommit string
 
 // The version of packer.
-const Version = "0.3.0"
+const Version = "0.3.4"
 
 // Any pre-release marker for the version. If this is "" (empty string),
 // then it means that it is a final release. Otherwise, this is the

panic.go

@@ -0,0 +1,62 @@
+package main
+
+import (
+	"fmt"
+	"github.com/mitchellh/panicwrap"
+	"io"
+	"os"
+	"strings"
+)
+
+// This is output if a panic happens.
+const panicOutput = `
+
+!!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
+
+Packer crashed! This is always indicative of a bug within Packer.
+A crash log has been placed at "crash.log" relative to your current
+working directory. It would be immensely helpful if you could please
+report the crash with Packer[1] so that we can fix this.
+
+[1]: https://github.com/mitchellh/packer/issues
+
+!!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
+`
+
+// panicHandler is what is called by panicwrap when a panic is encountered
+// within Packer. It is guaranteed to run after the resulting process has
+// exited so we can take the log file, add in the panic, and store it
+// somewhere locally.
+func panicHandler(logF *os.File) panicwrap.HandlerFunc {
+	return func(m string) {
+		// Write away just output this thing on stderr so that it gets
+		// shown in case anything below fails.
+		fmt.Fprintf(os.Stderr, fmt.Sprintf("%s\n", m))
+
+		// Create the crash log file where we'll write the logs
+		f, err := os.Create("crash.log")
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "Failed to create crash log file: %s", err)
+			return
+		}
+		defer f.Close()
+
+		// Seek the log file back to the beginning
+		if _, err = logF.Seek(0, 0); err != nil {
+			fmt.Fprintf(os.Stderr, "Failed to seek log file for crash: %s", err)
+			return
+		}
+
+		// Copy the contents to the crash file. This will include
+		// the panic that just happened.
+		if _, err = io.Copy(f, logF); err != nil {
+			fmt.Fprintf(os.Stderr, "Failed to write crash log: %s", err)
+			return
+		}
+
+		// Tell the user a crash occurred in some helpful way that
+		// they'll hopefully notice.
+		fmt.Printf("\n\n")
+		fmt.Println(strings.TrimSpace(panicOutput))
+	}
+}

plugin/command-inspect/main.go

@@ -0,0 +1,10 @@
+package main
+
+import (
+	"github.com/mitchellh/packer/command/inspect"
+	"github.com/mitchellh/packer/packer/plugin"
+)
+
+func main() {
+	plugin.ServeCommand(new(inspect.Command))
+}

plugin/command-inspect/main_test.go

@@ -0,0 +1 @@
+package main

post-processor/vagrant/aws.go

@@ -2,21 +2,22 @@ package vagrant
 
 import (
 	"fmt"
-	"github.com/mitchellh/mapstructure"
+	"github.com/mitchellh/packer/common"
 	"github.com/mitchellh/packer/packer"
 	"io/ioutil"
 	"log"
 	"os"
 	"path/filepath"
 	"strings"
-	"text/template"
 )
 
 type AWSBoxConfig struct {
+	common.PackerConfig `mapstructure:",squash"`
+
 	OutputPath          string `mapstructure:"output"`
 	VagrantfileTemplate string `mapstructure:"vagrantfile_template"`
 
-	PackerBuildName string `mapstructure:"packer_build_name"`
+	tpl *packer.ConfigTemplate
 }
 
 type AWSVagrantfileTemplate struct {
@@ -28,13 +29,36 @@ type AWSBoxPostProcessor struct {
 }
 
 func (p *AWSBoxPostProcessor) Configure(raws ...interface{}) error {
-	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &p.config)
-		if err != nil {
-			return err
+	md, err := common.DecodeConfig(&p.config, raws...)
+	if err != nil {
+		return err
+	}
+
+	p.config.tpl, err = packer.NewConfigTemplate()
+	if err != nil {
+		return err
+	}
+	p.config.tpl.UserVars = p.config.PackerUserVars
+
+	// Accumulate any errors
+	errs := common.CheckUnusedConfig(md)
+
+	validates := map[string]*string{
+		"output":               &p.config.OutputPath,
+		"vagrantfile_template": &p.config.VagrantfileTemplate,
+	}
+
+	for n, ptr := range validates {
+		if err := p.config.tpl.Validate(*ptr); err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Error parsing %s: %s", n, err))
 		}
 	}
 
+	if errs != nil && len(errs.Errors) > 0 {
+		return errs
+	}
+
 	return nil
 }
 
@@ -54,8 +78,11 @@ func (p *AWSBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifact
 	}
 
 	// Compile the output path
-	outputPath, err := ProcessOutputPath(p.config.OutputPath,
-		p.config.PackerBuildName, "aws", artifact)
+	outputPath, err := p.config.tpl.Process(p.config.OutputPath, &OutputPathTemplate{
+		ArtifactId: artifact.Id(),
+		BuildName:  p.config.PackerBuildName,
+		Provider:   "aws",
+	})
 	if err != nil {
 		return nil, false, err
 	}
@@ -93,8 +120,11 @@ func (p *AWSBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifact
 		vagrantfileContents = string(contents)
 	}
 
-	t := template.Must(template.New("vagrantfile").Parse(vagrantfileContents))
-	t.Execute(vf, tplData)
+	vagrantfileContents, err = p.config.tpl.Process(vagrantfileContents, tplData)
+	if err != nil {
+		return nil, false, fmt.Errorf("Error writing Vagrantfile: %s", err)
+	}
+	vf.Write([]byte(vagrantfileContents))
 	vf.Close()
 
 	// Create the metadata
@@ -104,7 +134,7 @@ func (p *AWSBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifact
 	}
 
 	// Compress the directory to the given output path
-	if err := DirToBox(outputPath, dir); err != nil {
+	if err := DirToBox(outputPath, dir, ui); err != nil {
 		err = fmt.Errorf("error creating box: %s", err)
 		return nil, false, err
 	}

post-processor/vagrant/post-processor.go

@@ -19,8 +19,6 @@ var builtins = map[string]string{
 
 type Config struct {
 	OutputPath string `mapstructure:"output"`
-
-	PackerBuildName string `mapstructure:"packer_build_name"`
 }
 
 type PostProcessor struct {

post-processor/vagrant/util.go

@@ -2,15 +2,14 @@ package vagrant
 
 import (
 	"archive/tar"
-	"bytes"
 	"compress/gzip"
 	"encoding/json"
+	"fmt"
 	"github.com/mitchellh/packer/packer"
 	"io"
 	"log"
 	"os"
 	"path/filepath"
-	"text/template"
 )
 
 // OutputPathTemplate is the structure that is availalable within the
@@ -45,7 +44,7 @@ func CopyContents(dst, src string) error {
 // DirToBox takes the directory and compresses it into a Vagrant-compatible
 // box. This function does not perform checks to verify that dir is
 // actually a proper box. This is an expected precondition.
-func DirToBox(dst, dir string) error {
+func DirToBox(dst, dir string, ui packer.Ui) error {
 	log.Printf("Turning dir into box: %s => %s", dir, dst)
 	dstF, err := os.Create(dst)
 	if err != nil {
@@ -93,6 +92,10 @@ func DirToBox(dst, dir string) error {
 			return err
 		}
 
+		if ui != nil {
+			ui.Message(fmt.Sprintf("Compressing: %s", header.Name))
+		}
+
 		if err := tarWriter.WriteHeader(header); err != nil {
 			return err
 		}
@@ -108,26 +111,6 @@ func DirToBox(dst, dir string) error {
 	return filepath.Walk(dir, tarWalk)
 }
 
-// ProcessOutputPath takes an output path template and executes it,
-// replacing variables with their respective values.
-func ProcessOutputPath(path string, buildName string, provider string, artifact packer.Artifact) (string, error) {
-	var buf bytes.Buffer
-
-	tplData := &OutputPathTemplate{
-		ArtifactId: artifact.Id(),
-		BuildName:  buildName,
-		Provider:   provider,
-	}
-
-	t, err := template.New("output").Parse(path)
-	if err != nil {
-		return "", err
-	}
-
-	err = t.Execute(&buf, tplData)
-	return buf.String(), err
-}
-
 // WriteMetadata writes the "metadata.json" file for a Vagrant box.
 func WriteMetadata(dir string, contents interface{}) error {
 	f, err := os.Create(filepath.Join(dir, "metadata.json"))

post-processor/vagrant/virtualbox.go

@@ -3,7 +3,7 @@ package vagrant
 import (
 	"errors"
 	"fmt"
-	"github.com/mitchellh/mapstructure"
+	"github.com/mitchellh/packer/common"
 	"github.com/mitchellh/packer/packer"
 	"io/ioutil"
 	"log"
@@ -11,14 +11,15 @@ import (
 	"path/filepath"
 	"regexp"
 	"strings"
-	"text/template"
 )
 
 type VBoxBoxConfig struct {
+	common.PackerConfig `mapstructure:",squash"`
+
 	OutputPath          string `mapstructure:"output"`
 	VagrantfileTemplate string `mapstructure:"vagrantfile_template"`
 
-	PackerBuildName string `mapstructure:"packer_build_name"`
+	tpl *packer.ConfigTemplate
 }
 
 type VBoxVagrantfileTemplate struct {
@@ -30,13 +31,36 @@ type VBoxBoxPostProcessor struct {
 }
 
 func (p *VBoxBoxPostProcessor) Configure(raws ...interface{}) error {
-	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &p.config)
-		if err != nil {
-			return err
+	md, err := common.DecodeConfig(&p.config, raws...)
+	if err != nil {
+		return err
+	}
+
+	p.config.tpl, err = packer.NewConfigTemplate()
+	if err != nil {
+		return err
+	}
+	p.config.tpl.UserVars = p.config.PackerUserVars
+
+	// Accumulate any errors
+	errs := common.CheckUnusedConfig(md)
+
+	validates := map[string]*string{
+		"output":               &p.config.OutputPath,
+		"vagrantfile_template": &p.config.VagrantfileTemplate,
+	}
+
+	for n, ptr := range validates {
+		if err := p.config.tpl.Validate(*ptr); err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Error parsing %s: %s", n, err))
 		}
 	}
 
+	if errs != nil && len(errs.Errors) > 0 {
+		return errs
+	}
+
 	return nil
 }
 
@@ -49,8 +73,11 @@ func (p *VBoxBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifac
 	}
 
 	// Compile the output path
-	outputPath, err := ProcessOutputPath(p.config.OutputPath,
-		p.config.PackerBuildName, "virtualbox", artifact)
+	outputPath, err := p.config.tpl.Process(p.config.OutputPath, &OutputPathTemplate{
+		ArtifactId: artifact.Id(),
+		BuildName:  p.config.PackerBuildName,
+		Provider:   "virtualbox",
+	})
 	if err != nil {
 		return nil, false, err
 	}
@@ -95,8 +122,11 @@ func (p *VBoxBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifac
 		vagrantfileContents = string(contents)
 	}
 
-	t := template.Must(template.New("vagrantfile").Parse(vagrantfileContents))
-	t.Execute(vf, tplData)
+	vagrantfileContents, err = p.config.tpl.Process(vagrantfileContents, tplData)
+	if err != nil {
+		return nil, false, fmt.Errorf("Error writing Vagrantfile: %s", err)
+	}
+	vf.Write([]byte(vagrantfileContents))
 	vf.Close()
 
 	// Create the metadata
@@ -113,7 +143,7 @@ func (p *VBoxBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifac
 
 	// Compress the directory to the given output path
 	ui.Message(fmt.Sprintf("Compressing box..."))
-	if err := DirToBox(outputPath, dir); err != nil {
+	if err := DirToBox(outputPath, dir, ui); err != nil {
 		return nil, false, err
 	}
 

post-processor/vagrant/vmware.go

@@ -2,19 +2,20 @@ package vagrant
 
 import (
 	"fmt"
-	"github.com/mitchellh/mapstructure"
+	"github.com/mitchellh/packer/common"
 	"github.com/mitchellh/packer/packer"
 	"io/ioutil"
 	"os"
 	"path/filepath"
-	"text/template"
 )
 
 type VMwareBoxConfig struct {
+	common.PackerConfig `mapstructure:",squash"`
+
 	OutputPath          string `mapstructure:"output"`
 	VagrantfileTemplate string `mapstructure:"vagrantfile_template"`
 
-	PackerBuildName string `mapstructure:"packer_build_name"`
+	tpl *packer.ConfigTemplate
 }
 
 type VMwareBoxPostProcessor struct {
@@ -22,20 +23,46 @@ type VMwareBoxPostProcessor struct {
 }
 
 func (p *VMwareBoxPostProcessor) Configure(raws ...interface{}) error {
-	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &p.config)
-		if err != nil {
-			return err
+	md, err := common.DecodeConfig(&p.config, raws...)
+	if err != nil {
+		return err
+	}
+
+	p.config.tpl, err = packer.NewConfigTemplate()
+	if err != nil {
+		return err
+	}
+	p.config.tpl.UserVars = p.config.PackerUserVars
+
+	// Accumulate any errors
+	errs := common.CheckUnusedConfig(md)
+
+	validates := map[string]*string{
+		"output":               &p.config.OutputPath,
+		"vagrantfile_template": &p.config.VagrantfileTemplate,
+	}
+
+	for n, ptr := range validates {
+		if err := p.config.tpl.Validate(*ptr); err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Error parsing %s: %s", n, err))
 		}
 	}
 
+	if errs != nil && len(errs.Errors) > 0 {
+		return errs
+	}
+
 	return nil
 }
 
 func (p *VMwareBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifact) (packer.Artifact, bool, error) {
 	// Compile the output path
-	outputPath, err := ProcessOutputPath(p.config.OutputPath,
-		p.config.PackerBuildName, "vmware", artifact)
+	outputPath, err := p.config.tpl.Process(p.config.OutputPath, &OutputPathTemplate{
+		ArtifactId: artifact.Id(),
+		BuildName:  p.config.PackerBuildName,
+		Provider:   "vmware",
+	})
 	if err != nil {
 		return nil, false, err
 	}
@@ -76,8 +103,11 @@ func (p *VMwareBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artif
 		}
 		defer vf.Close()
 
-		t := template.Must(template.New("vagrantfile").Parse(string(contents)))
-		t.Execute(vf, new(struct{}))
+		vagrantfileContents, err := p.config.tpl.Process(string(contents), nil)
+		if err != nil {
+			return nil, false, fmt.Errorf("Error writing Vagrantfile: %s", err)
+		}
+		vf.Write([]byte(vagrantfileContents))
 		vf.Close()
 	}
 
@@ -89,7 +119,7 @@ func (p *VMwareBoxPostProcessor) PostProcess(ui packer.Ui, artifact packer.Artif
 
 	// Compress the directory to the given output path
 	ui.Message(fmt.Sprintf("Compressing box..."))
-	if err := DirToBox(outputPath, dir); err != nil {
+	if err := DirToBox(outputPath, dir, ui); err != nil {
 		return nil, false, err
 	}
 

provisioner/file/provisioner.go

@@ -17,7 +17,7 @@ type config struct {
 	// The remote path where the local file will be uploaded to.
 	Destination string
 
-	tpl *common.Template
+	tpl *packer.ConfigTemplate
 }
 
 type Provisioner struct {
@@ -30,7 +30,7 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	p.config.tpl, err = common.NewTemplate()
+	p.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}

provisioner/salt-masterless/provisioner.go

@@ -27,7 +27,7 @@ type Config struct {
 	// Where files will be copied before moving to the /srv/salt directory
 	TempConfigDir string `mapstructure:"temp_config_dir"`
 
-	tpl *common.Template
+	tpl *packer.ConfigTemplate
 }
 
 type Provisioner struct {
@@ -40,7 +40,7 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	p.config.tpl, err = common.NewTemplate()
+	p.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}
@@ -137,6 +137,7 @@ func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
 func UploadLocalDirectory(localDir string, remoteDir string, comm packer.Communicator, ui packer.Ui) (err error) {
 	visitPath := func(localPath string, f os.FileInfo, err error) (err2 error) {
 		localRelPath := strings.Replace(localPath, localDir, "", 1)
+		localRelPath = strings.Replace(localRelPath, "\\", "/", -1)
 		remotePath := fmt.Sprintf("%s%s", remoteDir, localRelPath)
 		if f.IsDir() && f.Name() == ".git" {
 			return filepath.SkipDir

provisioner/shell/provisioner.go

@@ -12,6 +12,7 @@ import (
 	"log"
 	"os"
 	"strings"
+	"time"
 )
 
 const DefaultRemotePath = "/tmp/script.sh"
@@ -45,7 +46,13 @@ type config struct {
 	// can be used to inject the environment_vars into the environment.
 	ExecuteCommand string `mapstructure:"execute_command"`
 
-	tpl *common.Template
+	// The timeout for retrying to start the process. Until this timeout
+	// is reached, if the provisioner can't start a process, it retries.
+	// This can be set high to allow for reboots.
+	RawStartRetryTimeout string `mapstructure:"start_retry_timeout"`
+
+	startRetryTimeout time.Duration
+	tpl               *packer.ConfigTemplate
 }
 
 type Provisioner struct {
@@ -63,7 +70,7 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 		return err
 	}
 
-	p.config.tpl, err = common.NewTemplate()
+	p.config.tpl, err = packer.NewConfigTemplate()
 	if err != nil {
 		return err
 	}
@@ -84,6 +91,10 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 		p.config.InlineShebang = "/bin/sh"
 	}
 
+	if p.config.RawStartRetryTimeout == "" {
+		p.config.RawStartRetryTimeout = "5m"
+	}
+
 	if p.config.RemotePath == "" {
 		p.config.RemotePath = DefaultRemotePath
 	}
@@ -106,9 +117,10 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 	}
 
 	templates := map[string]*string{
-		"inline_shebang": &p.config.InlineShebang,
-		"script":         &p.config.Script,
-		"remote_path":    &p.config.RemotePath,
+		"inline_shebang":      &p.config.InlineShebang,
+		"script":              &p.config.Script,
+		"start_retry_timeout": &p.config.RawStartRetryTimeout,
+		"remote_path":         &p.config.RemotePath,
 	}
 
 	for n, ptr := range templates {
@@ -161,6 +173,14 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 		}
 	}
 
+	if p.config.RawStartRetryTimeout != "" {
+		p.config.startRetryTimeout, err = time.ParseDuration(p.config.RawStartRetryTimeout)
+		if err != nil {
+			errs = packer.MultiErrorAppend(
+				errs, fmt.Errorf("Failed parsing start_retry_timeout: %s", err))
+		}
+	}
+
 	if errs != nil && len(errs.Errors) > 0 {
 		return errs
 	}
@@ -217,7 +237,9 @@ func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
 		defer f.Close()
 
 		log.Printf("Uploading %s => %s", path, p.config.RemotePath)
-		err = comm.Upload(p.config.RemotePath, f)
+		err = p.retryable(func() error {
+			return comm.Upload(p.config.RemotePath, f)
+		})
 		if err != nil {
 			return fmt.Errorf("Error uploading shell script: %s", err)
 		}
@@ -239,8 +261,12 @@ func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
 
 		cmd := &packer.RemoteCmd{Command: command}
 		log.Printf("Executing command: %s", cmd.Command)
-		if err := cmd.StartWithUi(comm, ui); err != nil {
-			return fmt.Errorf("Failed executing command: %s", err)
+		err = p.retryable(func() error {
+			return cmd.StartWithUi(comm, ui)
+		})
+
+		if err != nil {
+			return err
 		}
 
 		if cmd.ExitStatus != 0 {
@@ -250,3 +276,29 @@ func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
 
 	return nil
 }
+
+// retryable will retry the given function over and over until a
+// non-error is returned.
+func (p *Provisioner) retryable(f func() error) error {
+	startTimeout := time.After(p.config.startRetryTimeout)
+	for {
+		var err error
+		if err = f(); err == nil {
+			return nil
+		}
+
+		// Create an error and log it
+		err = fmt.Errorf("Retryable error: %s", err)
+		log.Printf(err.Error())
+
+		// Check if we timed out, otherwise we retry. It is safe to
+		// retry since the only error case above is if the command
+		// failed to START.
+		select {
+		case <-startTimeout:
+			return err
+		default:
+			time.Sleep(2 * time.Second)
+		}
+	}
+}

scripts/build.sh

@@ -20,9 +20,19 @@ cd $DIR
 GIT_COMMIT=$(git rev-parse HEAD)
 GIT_DIRTY=$(test -n "`git status --porcelain`" && echo "+CHANGES" || true)
 
+# If we're building a race-enabled build, then set that up.
+if [ ! -z $PACKER_RACE ]; then
+    echo -e "${OK_COLOR}--> Building with race detection enabled${NO_COLOR}"
+    PACKER_RACE="-race"
+fi
+
+echo -e "${OK_COLOR}--> Installing dependencies to speed up builds...${NO_COLOR}"
+go get ./...
+
 # Compile the main Packer app
 echo -e "${OK_COLOR}--> Compiling Packer${NO_COLOR}"
 go build \
+    ${PACKER_RACE} \
     -ldflags "-X github.com/mitchellh/packer/packer.GitCommit ${GIT_COMMIT}${GIT_DIRTY}" \
     -v \
     -o bin/packer .
@@ -32,6 +42,7 @@ for PLUGIN in $(find ./plugin -mindepth 1 -maxdepth 1 -type d); do
     PLUGIN_NAME=$(basename ${PLUGIN})
     echo -e "${OK_COLOR}--> Compiling Plugin: ${PLUGIN_NAME}${NO_COLOR}"
     go build \
+        ${PACKER_RACE} \
         -ldflags "-X github.com/mitchellh/packer/packer.GitCommit ${GIT_COMMIT}${GIT_DIRTY}" \
         -v \
         -o bin/packer-${PLUGIN_NAME} ${PLUGIN}

scripts/dist.sh

@@ -60,15 +60,11 @@ waitAll() {
 trap "kill 0" SIGINT SIGTERM EXIT
 
 # Build our root project
-xc &
+xc
 
 # Build all the plugins
 for PLUGIN in $(find ./plugin -mindepth 1 -maxdepth 1 -type d); do
     PLUGIN_NAME=$(basename ${PLUGIN})
-    (
-    pushd ${PLUGIN}
-    xc
-    popd
     find ./pkg \
         -type f \
         -name ${PLUGIN_NAME} \
@@ -77,11 +73,8 @@ for PLUGIN in $(find ./plugin -mindepth 1 -maxdepth 1 -type d); do
         -type f \
         -name ${PLUGIN_NAME}.exe \
         -execdir mv ${PLUGIN_NAME}.exe packer-${PLUGIN_NAME}.exe ';'
-    ) &
 done
 
-waitAll
-
 # Zip all the packages
 mkdir -p ./pkg/${VERSIONDIR}/dist
 for PLATFORM in $(find ./pkg/${VERSIONDIR} -mindepth 1 -maxdepth 1 -type d); do

signal.go

@@ -26,7 +26,8 @@ func setupSignalHandlers(env packer.Environment) {
 
 		env.Ui().Error("Interrupt signal received twice. Forcefully exiting now.")
 
-		// Force kill all the plugins
+		// Force kill all the plugins, but mark that we're killing them
+		// first so that we don't get panics everywhere.
 		plugin.CleanupClients()
 		os.Exit(1)
 	}()

website/config.rb

@@ -45,7 +45,9 @@ set :images_dir, 'images'
 
 # Use the RedCarpet Markdown engine
 set :markdown_engine, :redcarpet
-set :markdown, :fenced_code_blocks => true
+set :markdown,
+    :fenced_code_blocks => true,
+    :with_toc_data => true
 
 # Build-specific configuration
 configure :build do

website/source/docs/builders/amazon-ebs.html.markdown

@@ -58,6 +58,12 @@ Required:
 
 Optional:
 
+* `ami_block_device_mappings` (array of block device mappings) - Add the block
+  device mappings to the AMI. The block device mappings allow for keys:
+  "device_name" (string), "virtual_name" (string), "snapshot_id" (string),
+  "volume_type" (string), "volume_size" (int), "delete_on_termination" (bool),
+  and "iops" (int).
+
 * `ami_description` (string) - The description to set for the resulting
   AMI(s). By default this description is empty.
 
@@ -77,6 +83,10 @@ Optional:
   [IAM instance profile](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html)
   to launch the EC2 instance with.
 
+* `launch_block_device_mappings` (array of block device mappings) - Add the
+  block device mappings to the launch instance. The block device mappings are
+  the same as `ami_block_device_mappings` above.
+
 * `security_group_id` (string) - The ID (_not_ the name) of the security
   group to assign to the instance. By default this is not set and Packer
   will automatically create a new temporary security group to allow SSH
@@ -130,6 +140,34 @@ the section above for more information on what environmental variables Packer
 will look for.
 </div>
 
+## AMI Block Device Mappings Example
+
+Here is an example using the optional AMI block device mappings. This will add
+the /dev/sdb and /dev/sdc block device mappings to the finished AMI.
+
+<pre class="prettyprint">
+{
+  "type": "amazon-ebs",
+  "access_key": "YOUR KEY HERE",
+  "secret_key": "YOUR SECRET KEY HERE",
+  "region": "us-east-1",
+  "source_ami": "ami-de0d9eb7",
+  "instance_type": "t1.micro",
+  "ssh_username": "ubuntu",
+  "ami_name": "packer-quick-start {{timestamp}}",
+  "ami_block_device_mappings": [
+      {
+          "device_name": "/dev/sdb",
+          "virtual_name": "ephemeral0"
+      },
+      {
+          "device_name": "/dev/sdc",
+          "virtual_name": "ephemeral1"
+      }
+  ]
+}
+</pre>
+
 ## Tag Example
 
 Here is an example using the optional AMI tags. This will add the tags

website/source/docs/builders/amazon-instance.html.markdown

@@ -73,6 +73,13 @@ Required:
 
 Optional:
 
+* `ami_block_device_mappings` (array of block device mappings) - Add the block
+  device mappings to the AMI. The block device mappings allow for keys:
+  "device_name" (string), "virtual_name" (string), "snapshot_id" (string),
+  "volume_type" (string), "volume_size" (int), "delete_on_termination" (bool),
+  and "iops" (int). See [amazon-ebs](/docs/builders/amazon-ebs.html) for an
+  example template.
+
 * `ami_description` (string) - The description to set for the resulting
   AMI(s). By default this description is empty.
 
@@ -108,6 +115,10 @@ Optional:
   [IAM instance profile](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html)
   to launch the EC2 instance with.
 
+* `launch_block_device_mappings` (array of block device mappings) - Add the
+  block device mappings to the launch instance. The block device mappings are
+  the same as `ami_block_device_mappings` above.
+
 * `security_group_id` (string) - The ID (_not_ the name) of the security
   group to assign to the instance. By default this is not set and Packer
   will automatically create a new temporary security group to allow SSH

website/source/docs/builders/virtualbox.html.markdown

@@ -82,6 +82,9 @@ Optional:
   be attached. The files listed in this configuration will all be put
   into the root directory of the floppy disk; sub-directories are not supported.
 
+* `format` (string) - Either "ovf" or "ova", this specifies the output
+  format of the exported virtual machine. This defaults to "ovf".
+
 * `guest_additions_path` (string) - The path on the guest virtual machine
   where the VirtualBox guest additions ISO will be uploaded. By default this
   is "VBoxGuestAdditions.iso" which should upload into the login directory
@@ -125,6 +128,12 @@ Optional:
   server to be on one port, make this minimum and maximum port the same.
   By default the values are 8000 and 9000, respectively.
 
+* `iso_urls` (array of strings) - Multiple URLs for the ISO to download.
+  Packer will try these in order. If anything goes wrong attempting to download
+  or while downloading a single URL, it will move on to the next. All URLs
+  must point to the same file (same checksum). By default this is empty
+  and `iso_url` is used. Only one of `iso_url` or `iso_urls` can be specified.
+
 * `output_directory` (string) - This is the path to the directory where the
   resulting virtual machine will be created. This may be relative or absolute.
   If relative, the path is relative to the working directory when `packer`

website/source/docs/builders/vmware.html.markdown

@@ -114,6 +114,12 @@ Optional:
   server to be on one port, make this minimum and maximum port the same.
   By default the values are 8000 and 9000, respectively.
 
+* `iso_urls` (array of strings) - Multiple URLs for the ISO to download.
+  Packer will try these in order. If anything goes wrong attempting to download
+  or while downloading a single URL, it will move on to the next. All URLs
+  must point to the same file (same checksum). By default this is empty
+  and `iso_url` is used. Only one of `iso_url` or `iso_urls` can be specified.
+
 * `output_directory` (string) - This is the path to the directory where the
   resulting virtual machine will be created. This may be relative or absolute.
   If relative, the path is relative to the working directory when `packer`

website/source/docs/command-line/inspect.html.markdown

@@ -0,0 +1,44 @@
+---
+layout: "docs"
+page_title: "Inspect - Command-Line"
+---
+
+# Command-Line: Inspect
+
+The `packer inspect` command takes a template and outputs the various components
+a template defines. This can help you quickly learn about a template without
+having to dive into the JSON itself.
+The command will tell you things like what variables a template accepts,
+the builders it defines, the provisioners it defines and the order they'll
+run, and more.
+
+This command is extra useful when used with
+[machine-readable output](/docs/command-line/machine-readable.html) enabled.
+The command outputs the components in a way that is parseable by machines.
+
+The command doesn't validate the actual configuration of the various
+components (that is what the `validate` command is for), but it will
+validate the syntax of your template by necessity.
+
+## Usage Example
+
+Given a basic template, here is an example of what the output might
+look like:
+
+```
+$ packer inspect template.json
+Variables and their defaults:
+
+  aws_access_key =
+  aws_secret_key =
+
+Builders:
+
+  amazon-ebs
+  amazon-instance
+  virtualbox
+
+Provisioners:
+
+  shell
+```

website/source/docs/extend/builder.html.markdown

@@ -1,5 +1,6 @@
 ---
 layout: "docs"
+page_title: "Custom Builder - Extend Packer"
 ---
 
 # Custom Builder Development

website/source/docs/extend/plugins.html.markdown

@@ -1,5 +1,6 @@
 ---
 layout: "docs"
+page_title: "Packer Plugins - Extend Packer"
 ---
 
 # Packer Plugins

website/source/docs/machine-readable/command-inspect.html.markdown

@@ -0,0 +1,56 @@
+---
+layout: "docs_machine_readable"
+page_title: "Command: inspect - Machine-Readable Reference"
+---
+
+# Inspect Command Types
+
+These are the machine-readable types that exist as part of the output
+of `packer inspect`.
+
+<dl>
+	<dt>template-variable (2)</dt>
+	<dd>
+		<p>
+		A <a href="/docs/templates/user-variables.html">user variable</a>
+		defined within the template.
+		</p>
+
+		<p>
+		<strong>Data 1: name</strong> - Name of the variable.
+		</p>
+
+		<p>
+		<strong>Data 2: default</strong> - The default value of the
+		variable.
+		</p>
+	</dd>
+
+	<dt>template-builder (2)</dt>
+	<dd>
+		<p>
+		A builder defined within the template
+		</p>
+
+		<p>
+		<strong>Data 1: name</strong> - The name of the builder.
+		</p>
+		<p>
+		<strong>Data 2: type</strong> - The type of the builder. This will
+		generally be the same as the name unless you explicitly override
+		the name.
+		</p>
+	</dd>
+
+	<dt>template-provisioner (1)</dt>
+	<dd>
+		<p>
+		A provisioner defined within the template. Multiple of these may
+		exist. If so, they are outputted in the order they would run.
+		</p>
+
+		<p>
+		<strong>Data 1: name</strong> - The name/type of the provisioner.
+		</p>
+	</dd>
+</dl>

website/source/docs/provisioners/shell.html.markdown

@@ -53,8 +53,7 @@ Optional parameters:
 
 * `execute_command` (string) - The command to use to execute the script.
   By default this is `chmod +x {{ .Path }}; {{ .Vars }} {{ .Path }}`. The value of this is
-  treated as [configuration template](/docs/templates/configuration-
-  templates.html). There are two available variables: `Path`, which is
+  treated as [configuration template](/docs/templates/configuration-templates.html). There are two available variables: `Path`, which is
   the path to the script to run, and `Vars`, which is the list of
   `environment_vars`, if configured.
 
@@ -67,6 +66,12 @@ Optional parameters:
   in the machine. This defaults to "/tmp/script.sh". This value must be
   a writable location and any parent directories must already exist.
 
+* `start_retry_timeout` (string) - The amount of time to attempt to
+  _start_ the remote process. By default this is "5m" or 5 minutes. This
+  setting exists in order to deal with times when SSH may restart, such as
+  a system reboot. Set this to a higher value if reboots take a longer
+  amount of time.
+
 ## Execute Command Example
 
 To many new users, the `execute_command` is puzzling. However, it provides
@@ -102,3 +107,56 @@ defines certain commonly useful environmental variables:
 * `PACKER_BUILDER_TYPE` is the type of the builder that was used to create
   the machine that the script is running on. This is useful if you want to
   run only certain parts of the script on systems built with certain builders.
+
+## Handling Reboots
+
+Provisioning sometimes involves restarts, usually when updating the operating
+system. Packer is able to tolerate restarts via the shell provisioner.
+
+Packer handles this by retrying to start scripts for a period of time
+before failing. This allows time for the machine to start up and be ready
+to run scripts. The amount of time the provisioner will wait is configured
+using `start_retry_timeout`, which defaults to a few minutes.
+
+Sometimes, when executing a command like `reboot`, the shell script will
+return and Packer will start executing the next one before SSH actually
+quits and the machine restarts. For this, put a long `sleep` after the
+reboot so that SSH will eventually be killed automatically:
+
+```
+reboot
+sleep 60
+```
+
+## Troubleshooting
+
+*My shell script doesn't work correctly on Ubuntu*
+
+* On Ubuntu the /bin/sh shell is
+[dash](http://en.wikipedia.org/wiki/Debian_Almquist_shell). If your script has
+[bash](http://en.wikipedia.org/wiki/Bash_(Unix_shell\)) specific commands in it
+then put `#!/bin/bash` at the top of your script. Differences
+between dash and bash can be found on the [DashAsBinSh](https://wiki.ubuntu.com/DashAsBinSh) Ubuntu wiki page.
+
+*My shell works when I login but fails with the shell provisioner*
+
+* See the above tip. More than likely your login shell is using /bin/bash
+while the provisioner is using /bin/sh.
+
+*How do I tell what my shell script is doing?*
+
+* Adding a `-x` flag to the shebang at the top of the script (`#!/bin/sh -x`)
+will echo the script statements as it is executing.
+
+*My builds don't always work the same*
+
+* Some distributions start the SSH daemon before other core services which
+can create race conditions. Your first provisoner can tell the machine to
+wait until it completely boots.
+
+<pre class="prettyprint">
+    {
+       "type": "script",
+       "inline": [ "sleep 10" ]
+    }
+</pre>

website/source/layouts/docs.erb

@@ -12,6 +12,7 @@
 			<li><a href="/docs/command-line/introduction.html">Introduction</a></li>
 			<li><a href="/docs/command-line/build.html">Build</a></li>
 			<li><a href="/docs/command-line/fix.html">Fix</a></li>
+			<li><a href="/docs/command-line/inspect.html">Inspect</a></li>
 			<li><a href="/docs/command-line/validate.html">Validate</a></li>
 			<li><a href="/docs/command-line/machine-readable.html">Machine-Readable Output</a></li>
 		</ul>

website/source/layouts/docs_machine_readable.erb

@@ -7,6 +7,7 @@
 			<li><a href="/docs/index.html">&laquo; Back to Docs</a></li>
 			<li><a href="/docs/machine-readable/general.html">General Types</a></li>
 			<li><a href="/docs/machine-readable/command-build.html">Command: build</a></li>
+			<li><a href="/docs/machine-readable/command-inspect.html">Command: inspect</a></li>
 			<li><a href="/docs/machine-readable/command-version.html">Command: version</a></li>
 		</ul>
 	<% end %>