v0.2.0

This unifies all the builders so that they all wait for SSH using a common
step available in builder/common. This common step has the best practices of
every previously unique step, such as retrying SSH handshakes, limiting
the number of SSH handshakes, timeouts, and so on.

.gitignore

@@ -1,5 +1,6 @@
 /bin
 /local
 /pkg
+/src
 /website/.sass-cache
 /website/build

CHANGELOG.md

@@ -1,4 +1,133 @@
-## 0.1.2 (unreleased)
+## 0.2.0 (July 16, 2013)
+
+BACKWARDS INCOMPATIBILITIES:
+
+* "iso_md5" in the virtualbox and vmware builders is replaced with
+  "iso_checksum" and "iso_checksum_type" (with the latter set to "md5").
+  See the announce below on `packer fix` to automatically fix your templates.
+
+FEATURES:
+
+* **NEW COMMAND:** `packer fix` will attempt to fix templates from older
+  versions of Packer that are now broken due to backwards incompatibilities.
+  This command will fix the backwards incompatibilities introduced in this
+  version.
+* Amazon EBS builder can now optionally use a pre-made security group
+  instead of randomly generating one.
+* DigitalOcean API key and client IDs can now be passed in as
+  environmental variables. See the documentatin for more details.
+* VirtualBox and VMware can now have `floppy_files` specified to attach
+  floppy disks when booting. This allows for unattended Windows installs.
+* `packer build` has a new `-force` flag that forces the removal of
+  existing artifacts if they exist. [GH-173]
+* You can now log to a file (instead of just stderr) by setting the
+  `PACKER_LOG_FILE` environmental variable. [GH-168]
+* Checksums other than MD5 can now be used. SHA1 and SHA256 can also
+  be used. See the documentation on `iso_checksum_type` for more info. [GH-175]
+
+IMPROVEMENTS:
+
+* core: invalid keys in configuration are now considered validation
+  errors. [GH-104]
+* core: all builders now share a common SSH connection core, improving
+  SSH reliability over all the builders.
+* amazon-ebs: Credentials will come from IAM role if available. [GH-160]
+* amazon-ebs: Verify the source AMI is EBS-backed before launching. [GH-169]
+* shell provisioner: the build name and builder type are available in
+  the `PACKER_BUILD_NAME` and `PACKER_BUILDER_TYPE` env vars by default,
+  respectively. [GH-154]
+* vmware: error if shutdown command has non-zero exit status.
+
+BUG FIXES:
+
+* core: UI messages are now properly prefixed with spaces again.
+* core: If SSH connection ends, re-connection attempts will take
+  place. [GH-152]
+* virtualbox: "paused" doesn't mean the VM is stopped, improving
+  shutdown detection.
+* vmware: error if guest IP could not be detected. [GH-189]
+
+## 0.1.5 (July 7, 2013)
+
+FEATURES:
+
+* "file" uploader will upload files from the machine running Packer to the
+  remote machine.
+* VirtualBox guest additions URL and checksum can now be specified, allowing
+  the VirtualBox builder to have the ability to be used completely offline.
+
+IMPROVEMENTS:
+
+* core: If SCP is not available, a more descriptive error message
+  is shown telling the user. [GH-127]
+* shell: Scripts are now executed by default according to their shebang,
+  not with `/bin/sh`. [GH-105]
+* shell: You can specify what interpreter you want inline scripts to
+  run with `inline_shebang`.
+* virtualbox: Delete the packer-made SSH port forwarding prior to
+  exporting the VM.
+
+BUG FIXES:
+
+* core: Non-200 response codes on downloads now show proper errors.
+  [GH-141]
+* amazon-ebs: SSH handshake is retried. [GH-130]
+* vagrant: The `BuildName` template propery works properly in
+  the output path.
+* vagrant: Properly configure the provider-specific post-processors so
+  things like `vagrantfile_template` work. [GH-129]
+* vagrant: Close filehandles when copying files so Windows can
+  rename files. [GH-100]
+
+## 0.1.4 (July 2, 2013)
+
+FEATURES:
+
+* virtualbox: Can now be built headless with the "Headless" option. [GH-99]
+* virtualbox: <wait5> and <wait10> codes for waiting 5 and 10 seconds
+  during the boot sequence, respectively. [GH-97]
+* vmware: Can now be built headless with the "Headless" option. [GH-99]
+* vmware: <wait5> and <wait10> codes for waiting 5 and 10 seconds
+  during the boot sequence, respectively. [GH-97]
+* vmware: Disks are defragmented and compacted at the end of the build.
+  This can be disabled using "skip_compaction"
+
+IMPROVEMENTS:
+
+* core: Template syntax errors now show line and character number. [GH-56]
+* amazon-ebs: Access key and secret access key default to
+  environmental variables. [GH-40]
+* virtualbox: Send password for keyboard-interactive auth [GH-121]
+* vmware: Send password for keyboard-interactive auth [GH-121]
+
+BUG FIXES:
+
+* vmware: Wait until shut down cleans up properly to avoid corrupt
+  disk files [GH-111]
+
+## 0.1.3 (July 1, 2013)
+
+FEATURES:
+
+* The VMware builder can now upload the VMware tools for you into
+  the VM. This is opt-in, you must specify the `tools_upload_flavor`
+  option. See the website for more documentation.
+
+IMPROVEMENTS:
+
+* digitalocean: Errors contain human-friendly error messages. [GH-85]
+
+BUG FIXES:
+
+* core: More plugin server fixes that avoid hangs on OS X 10.7 [GH-87]
+* vagrant: AWS boxes will keep the AMI artifact around [GH-55]
+* virtualbox: More robust version parsing for uploading guest additions. [GH-69]
+* virtualbox: Output dir and VM name defaults depend on build name,
+  avoiding collisions. [GH-91]
+* vmware: Output dir and VM name defaults depend on build name,
+  avoiding collisions. [GH-91]
+
+## 0.1.2 (June 29, 2013)
 
 IMPROVEMENTS:
 

CONTRIBUTING.md

@@ -0,0 +1,67 @@
+# Contributing to Packer
+
+**First:** if you're unsure or afraid of _anything_, just ask
+or submit the issue or pull request anyways. You won't be yelled at for
+giving your best effort. The worst that can happen is that you'll be
+politely asked to change something. We appreciate any sort of contributions,
+and don't want a wall of rules to get in the way of that.
+
+However, for those individuals who want a bit more guidance on the
+best way to contribute to the project, read on. This document will cover
+what we're looking for. By addressing all the points we're looking for,
+it raises the chances we can quickly merge or address your contributions.
+
+## Issues
+
+### Reporting an Issue
+
+* Make sure you test against the latest released version. It is possible
+  we already fixed the bug you're experiencing.
+
+* Provide a reproducible test case. If a contributor can't reproduce an
+  issue, then it dramatically lowers the chances it'll get fixed. And in
+  some cases, the issue will eventually be closed.
+
+* Respond promptly to any questions made by the Packer team to your issue.
+  Stale issues will be closed.
+
+### Issue Lifecycle
+
+1. The issue is reported.
+
+2. The issue is verified and categorized by a Packer collaborator.
+   Categorization is done via tags. For example, bugs are marked as "bugs"
+   and easy fixes are marked as "easy".
+
+3. Unless it is critical, the issue is left for a period of time (sometimes
+   many weeks), giving outside contributors a chance to address the issue.
+
+4. The issue is addressed in a pull request or commit. The issue will be
+   referenced in the commit message so that the code that fixes it is clearly
+   linked.
+
+5. The issue is closed.
+
+## Setting up Go to work on Packer
+
+If you have never worked with Go before, you will have to complete the
+following steps in order to be able to compile and test Packer.
+
+1. Install Go. On a Mac, you can `brew install go`.
+
+2. Set and export the `GOPATH` environment variable. For example, you can
+   add `export GOPATH=$HOME/Documents/golang` to your `.bash_profile`.
+
+3. Download the Packer source (and its dependencies) by running
+   `go get github.com/mitchellh/packer`. This will download the Packer
+   source to `$GOPATH/src/github.com/mitchellh/packer`.
+
+4. Make your changes to the Packer source. You can run `make` from the main
+   source directory to recompile all the binaries. Any compilation errors
+   will be shown when the binaries are rebuilding.
+
+5. Test your changes by running `make test` and then running
+   `$GOPATH/src/github.com/mitchellh/packer/bin/packer` to build a machine.
+
+6. If everything works well and the tests pass, run `go fmt` on your code
+   before submitting a pull request.

Makefile

@@ -1,7 +1,7 @@
-NO_COLOR=\x1b[0m
-OK_COLOR=\x1b[32;01m
-ERROR_COLOR=\x1b[31;01m
-WARN_COLOR=\x1b[33;01m
+NO_COLOR=\033[0m
+OK_COLOR=\033[32;01m
+ERROR_COLOR=\033[31;01m
+WARN_COLOR=\033[33;01m
 
 all:
 	@mkdir -p bin/
@@ -15,8 +15,7 @@ format:
 
 test:
 	@echo "$(OK_COLOR)==> Testing Packer...$(NO_COLOR)"
-	@go list -f '{{range .TestImports}}{{.}}\
-		{{end}}' ./... | xargs -n1 go get -d
+	@go list -f '{{range .TestImports}}{{.}} {{end}}' ./... | xargs -n1 go get -d
 	go test ./...
 
 .PHONY: all format test

README.md

@@ -11,7 +11,7 @@ Packer is lightweight, runs on every major operating system, and is highly
 performant, creating machine images for multiple platforms in parallel.
 Packer comes out of the box with support for creating AMIs (EC2), VMware
 images, and VirtualBox images. Support for more platforms can be added via
-plugins. 
+plugins.
 
 The images that Packer creates can easily be turned into
 [Vagrant](http://www.vagrantup.com) boxes.
@@ -70,8 +70,15 @@ http://www.packer.io/docs
 ## Developing Packer
 
 If you wish to work on Packer itself, you'll first need [Go](http://golang.org)
-installed (version 1.1+ is _required_). Next, clone this repository then just type `make`.
-In a few moments, you'll have a working `packer` executable:
+installed (version 1.1+ is _required_). Make sure you have Go properly installed,
+including setting up your [GOPATH](http://golang.org/doc/code.html#GOPATH).
+
+For some additional dependencies, Go needs [Mercurial](http://mercurial.selenic.com/)
+to be installed. Packer itself doesn't require this but a dependency of a
+dependency does.
+
+Next, clone this repository into `$GOPATH/src/github.com/mitchellh/packer` and
+then just type `make`. In a few moments, you'll have a working `packer` executable:
 
 ```
 $ make
@@ -80,6 +87,6 @@ $ bin/packer
 ...
 ```
 
-You can run tests by typing `make test`. 
+You can run tests by typing `make test`.
 
 This will run tests for Packer core along with all the core builders and commands and such that come with Packer.

builder/amazon/ebs/artifact.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"fmt"

builder/amazon/ebs/artifact_test.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"cgl.tideland.biz/asserts"

builder/amazon/ebs/builder.go

@@ -3,7 +3,7 @@
 //
 // In general, there are two types of AMIs that can be created: ebs-backed or
 // instance-store. This builder _only_ builds ebs-backed images.
-package amazonebs
+package ebs
 
 import (
 	"errors"
@@ -15,6 +15,8 @@ import (
 	"github.com/mitchellh/packer/builder/common"
 	"github.com/mitchellh/packer/packer"
 	"log"
+	"sort"
+	"strings"
 	"text/template"
 	"time"
 )
@@ -28,18 +30,21 @@ type config struct {
 	SecretKey string `mapstructure:"secret_key"`
 
 	// Information for the source instance
-	Region       string
-	SourceAmi    string `mapstructure:"source_ami"`
-	InstanceType string `mapstructure:"instance_type"`
-	SSHUsername  string `mapstructure:"ssh_username"`
-	SSHPort      int    `mapstructure:"ssh_port"`
-	SSHTimeout   time.Duration
+	Region          string
+	SourceAmi       string `mapstructure:"source_ami"`
+	InstanceType    string `mapstructure:"instance_type"`
+	SSHUsername     string `mapstructure:"ssh_username"`
+	SSHPort         int    `mapstructure:"ssh_port"`
+	SecurityGroupId string `mapstructure:"security_group_id"`
 
 	// Configuration of the resulting AMI
 	AMIName string `mapstructure:"ami_name"`
 
 	PackerDebug   bool   `mapstructure:"packer_debug"`
 	RawSSHTimeout string `mapstructure:"ssh_timeout"`
+
+	// Unexported fields that are calculated from others
+	sshTimeout time.Duration
 }
 
 type Builder struct {
@@ -48,15 +53,38 @@ type Builder struct {
 }
 
 func (b *Builder) Prepare(raws ...interface{}) error {
-	var err error
+	var md mapstructure.Metadata
+	decoderConfig := &mapstructure.DecoderConfig{
+		Metadata: &md,
+		Result:   &b.config,
+	}
+
+	decoder, err := mapstructure.NewDecoder(decoderConfig)
+	if err != nil {
+		return err
+	}
 
 	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &b.config)
+		err := decoder.Decode(raw)
 		if err != nil {
 			return err
 		}
 	}
 
+	// Accumulate any errors
+	errs := make([]error, 0)
+
+	// Unused keys are errors
+	if len(md.Unused) > 0 {
+		sort.Strings(md.Unused)
+		for _, unused := range md.Unused {
+			if unused != "type" && !strings.HasPrefix(unused, "packer_") {
+				errs = append(
+					errs, fmt.Errorf("Unknown configuration key: %s", unused))
+			}
+		}
+	}
+
 	if b.config.SSHPort == 0 {
 		b.config.SSHPort = 22
 	}
@@ -66,16 +94,6 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 	}
 
 	// Accumulate any errors
-	errs := make([]error, 0)
-
-	if b.config.AccessKey == "" {
-		errs = append(errs, errors.New("An access_key must be specified"))
-	}
-
-	if b.config.SecretKey == "" {
-		errs = append(errs, errors.New("A secret_key must be specified"))
-	}
-
 	if b.config.SourceAmi == "" {
 		errs = append(errs, errors.New("A source_ami must be specified"))
 	}
@@ -94,7 +112,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		errs = append(errs, errors.New("An ssh_username must be specified"))
 	}
 
-	b.config.SSHTimeout, err = time.ParseDuration(b.config.RawSSHTimeout)
+	b.config.sshTimeout, err = time.ParseDuration(b.config.RawSSHTimeout)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing ssh_timeout: %s", err))
 	}
@@ -122,7 +140,11 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		panic("region not found")
 	}
 
-	auth := aws.Auth{b.config.AccessKey, b.config.SecretKey}
+	auth, err := aws.GetAuth(b.config.AccessKey, b.config.SecretKey)
+	if err != nil {
+		return nil, err
+	}
+
 	ec2conn := ec2.New(auth, region)
 
 	// Setup the state bag and initial state for the steps
@@ -137,8 +159,12 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		&stepKeyPair{},
 		&stepSecurityGroup{},
 		&stepRunSourceInstance{},
-		&stepConnectSSH{},
-		&stepProvision{},
+		&common.StepConnectSSH{
+			SSHAddress:     sshAddress,
+			SSHConfig:      sshConfig,
+			SSHWaitTimeout: b.config.sshTimeout,
+		},
+		&common.StepProvision{},
 		&stepStopInstance{},
 		&stepCreateAMI{},
 	}

builder/amazon/ebs/builder_test.go

@@ -1,10 +1,20 @@
-package amazonebs
+package ebs
 
 import (
 	"github.com/mitchellh/packer/packer"
+	"os"
 	"testing"
 )
 
+func init() {
+	// Clear out the AWS access key env vars so they don't
+	// affect our tests.
+	os.Setenv("AWS_ACCESS_KEY_ID", "")
+	os.Setenv("AWS_ACCESS_KEY", "")
+	os.Setenv("AWS_SECRET_ACCESS_KEY", "")
+	os.Setenv("AWS_SECRET_KEY", "")
+}
+
 func testConfig() map[string]interface{} {
 	return map[string]interface{}{
 		"access_key":    "foo",
@@ -37,30 +47,6 @@ func TestBuilder_Prepare_BadType(t *testing.T) {
 	}
 }
 
-func TestBuilderPrepare_AccessKey(t *testing.T) {
-	var b Builder
-	config := testConfig()
-
-	// Test good
-	config["access_key"] = "foo"
-	err := b.Prepare(config)
-	if err != nil {
-		t.Fatalf("should not have error: %s", err)
-	}
-
-	if b.config.AccessKey != "foo" {
-		t.Errorf("access key invalid: %s", b.config.AccessKey)
-	}
-
-	// Test bad
-	delete(config, "access_key")
-	b = Builder{}
-	err = b.Prepare(config)
-	if err == nil {
-		t.Fatal("should have error")
-	}
-}
-
 func TestBuilderPrepare_AMIName(t *testing.T) {
 	var b Builder
 	config := testConfig()
@@ -113,6 +99,18 @@ func TestBuilderPrepare_InstanceType(t *testing.T) {
 	}
 }
 
+func TestBuilderPrepare_InvalidKey(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Add a random key
+	config["i_should_not_be_valid"] = true
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+}
+
 func TestBuilderPrepare_Region(t *testing.T) {
 	var b Builder
 	config := testConfig()
@@ -145,30 +143,6 @@ func TestBuilderPrepare_Region(t *testing.T) {
 	}
 }
 
-func TestBuilderPrepare_SecretKey(t *testing.T) {
-	var b Builder
-	config := testConfig()
-
-	// Test good
-	config["secret_key"] = "foo"
-	err := b.Prepare(config)
-	if err != nil {
-		t.Fatalf("should not have error: %s", err)
-	}
-
-	if b.config.SecretKey != "foo" {
-		t.Errorf("secret key invalid: %s", b.config.SecretKey)
-	}
-
-	// Test bad
-	delete(config, "secret_key")
-	b = Builder{}
-	err = b.Prepare(config)
-	if err == nil {
-		t.Fatal("should have error")
-	}
-}
-
 func TestBuilderPrepare_SourceAmi(t *testing.T) {
 	var b Builder
 	config := testConfig()

builder/amazon/ebs/hook.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 // This hook is fired prior to launching the EC2 instance.
 const HookPreLaunch = "amazonebs_pre_launch"

builder/amazon/ebs/instance.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"fmt"

builder/amazon/ebs/ssh.go

@@ -0,0 +1,31 @@
+package ebs
+
+import (
+	gossh "code.google.com/p/go.crypto/ssh"
+	"fmt"
+	"github.com/mitchellh/goamz/ec2"
+	"github.com/mitchellh/packer/communicator/ssh"
+)
+
+func sshAddress(state map[string]interface{}) (string, error) {
+	config := state["config"].(config)
+	instance := state["instance"].(*ec2.Instance)
+	return fmt.Sprintf("%s:%d", instance.DNSName, config.SSHPort), nil
+}
+
+func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
+	config := state["config"].(config)
+	privateKey := state["privateKey"].(string)
+
+	keyring := new(ssh.SimpleKeychain)
+	if err := keyring.AddPEMKey(privateKey); err != nil {
+		return nil, fmt.Errorf("Error setting up SSH config: %s", err)
+	}
+
+	return &gossh.ClientConfig{
+		User: config.SSHUsername,
+		Auth: []gossh.ClientAuth{
+			gossh.ClientAuthKeyring(keyring),
+		},
+	}, nil
+}

builder/amazon/ebs/step_create_ami.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"bytes"

builder/amazon/ebs/step_keypair.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"cgl.tideland.biz/identifier"

builder/amazon/ebs/step_run_source_instance.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"fmt"
@@ -29,6 +29,19 @@ func (s *stepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
 	}
 
 	ui.Say("Launching a source AWS instance...")
+	imageResp, err := ec2conn.Images([]string{config.SourceAmi}, ec2.NewFilter())
+	if err != nil {
+		state["error"] = fmt.Errorf("There was a problem with the source AMI: %s", err)
+		return multistep.ActionHalt
+	}
+
+	if imageResp.Images[0].RootDeviceType != "ebs" {
+		state["error"] = fmt.Errorf(
+			"The provided source AMI is instance-store based. The\n" +
+				"amazon-ebs bundler can only work with EBS based AMIs.")
+		return multistep.ActionHalt
+	}
+
 	runResp, err := ec2conn.RunInstances(runOpts)
 	if err != nil {
 		err := fmt.Errorf("Error launching source instance: %s", err)
@@ -40,10 +53,10 @@ func (s *stepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
 	s.instance = &runResp.Instances[0]
 	log.Printf("instance id: %s", s.instance.InstanceId)
 
-	ui.Say("Waiting for instance to become ready...")
+	ui.Say(fmt.Sprintf("Waiting for instance (%s) to become ready...", s.instance.InstanceId))
 	s.instance, err = waitForState(ec2conn, s.instance, []string{"pending"}, "running")
 	if err != nil {
-		err := fmt.Errorf("Error waiting for instance to become ready: %s", err)
+		err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", s.instance.InstanceId, err)
 		state["error"] = err
 		ui.Error(err.Error())
 		return multistep.ActionHalt

builder/amazon/ebs/step_security_group.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"cgl.tideland.biz/identifier"
@@ -19,6 +19,12 @@ func (s *stepSecurityGroup) Run(state map[string]interface{}) multistep.StepActi
 	ec2conn := state["ec2"].(*ec2.EC2)
 	ui := state["ui"].(packer.Ui)
 
+	if config.SecurityGroupId != "" {
+		log.Printf("Using specified security group: %s", config.SecurityGroupId)
+		state["securityGroupId"] = config.SecurityGroupId
+		return multistep.ActionContinue
+	}
+
 	// Create the group
 	ui.Say("Creating temporary security group for this instance...")
 	groupName := fmt.Sprintf("packer %s", hex.EncodeToString(identifier.NewUUID().Raw()))

builder/amazon/ebs/step_stop_instance.go

@@ -1,4 +1,4 @@
-package amazonebs
+package ebs
 
 import (
 	"fmt"

builder/amazonebs/step_connect_ssh.go

@@ -1,125 +0,0 @@
-package amazonebs
-
-import (
-	gossh "code.google.com/p/go.crypto/ssh"
-	"errors"
-	"fmt"
-	"github.com/mitchellh/goamz/ec2"
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/communicator/ssh"
-	"github.com/mitchellh/packer/packer"
-	"log"
-	"net"
-	"time"
-)
-
-type stepConnectSSH struct {
-	conn net.Conn
-}
-
-func (s *stepConnectSSH) Run(state map[string]interface{}) multistep.StepAction {
-	config := state["config"].(config)
-	instance := state["instance"].(*ec2.Instance)
-	privateKey := state["privateKey"].(string)
-	ui := state["ui"].(packer.Ui)
-
-	// Build the keyring for authentication. This stores the private key
-	// we'll use to authenticate.
-	keyring := &ssh.SimpleKeychain{}
-	err := keyring.AddPEMKey(privateKey)
-	if err != nil {
-		err := fmt.Errorf("Error setting up SSH config: %s", err)
-		state["error"] = err
-		ui.Error(err.Error())
-		return multistep.ActionHalt
-	}
-
-	// Build the actual SSH client configuration
-	sshConfig := &gossh.ClientConfig{
-		User: config.SSHUsername,
-		Auth: []gossh.ClientAuth{
-			gossh.ClientAuthKeyring(keyring),
-		},
-	}
-
-	// Start trying to connect to SSH
-	connected := make(chan bool, 1)
-	connectQuit := make(chan bool, 1)
-	defer func() {
-		connectQuit <- true
-	}()
-
-	go func() {
-		var err error
-
-		ui.Say("Connecting to the instance via SSH...")
-		attempts := 0
-		for {
-			select {
-			case <-connectQuit:
-				return
-			default:
-			}
-
-			attempts += 1
-			log.Printf(
-				"Opening TCP conn for SSH to %s:%d (attempt %d)",
-				instance.DNSName, config.SSHPort, attempts)
-			s.conn, err = net.Dial("tcp", fmt.Sprintf("%s:%d", instance.DNSName, config.SSHPort))
-			if err == nil {
-				break
-			}
-
-			// A brief sleep so we're not being overly zealous attempting
-			// to connect to the instance.
-			time.Sleep(500 * time.Millisecond)
-		}
-
-		connected <- true
-	}()
-
-	log.Printf("Waiting up to %s for SSH connection", config.SSHTimeout)
-	timeout := time.After(config.SSHTimeout)
-
-ConnectWaitLoop:
-	for {
-		select {
-		case <-connected:
-			// We connected. Just break the loop.
-			break ConnectWaitLoop
-		case <-timeout:
-			err := errors.New("Timeout waiting for SSH to become available.")
-			state["error"] = err
-			ui.Error(err.Error())
-			return multistep.ActionHalt
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				log.Println("Interrupt detected, quitting waiting for SSH.")
-				return multistep.ActionHalt
-			}
-		}
-	}
-
-	var comm packer.Communicator
-	if err == nil {
-		comm, err = ssh.New(s.conn, sshConfig)
-	}
-
-	if err != nil {
-		err := fmt.Errorf("Error connecting to SSH: %s", err)
-		state["error"] = err
-		ui.Error(err.Error())
-		return multistep.ActionHalt
-	}
-
-	// Set the communicator on the state bag so it can be used later
-	state["communicator"] = comm
-
-	return multistep.ActionContinue
-}
-
-func (s *stepConnectSSH) Cleanup(map[string]interface{}) {
-	if s.conn != nil {
-		s.conn.Close()
-	}
-}

builder/amazonebs/step_provision.go

@@ -1,25 +0,0 @@
-package amazonebs
-
-import (
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/packer"
-	"log"
-)
-
-type stepProvision struct{}
-
-func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
-	comm := state["communicator"].(packer.Communicator)
-	hook := state["hook"].(packer.Hook)
-	ui := state["ui"].(packer.Ui)
-
-	log.Println("Running the provision hook")
-	if err := hook.Run(packer.HookProvision, ui, comm, nil); err != nil {
-		state["error"] = err
-		return multistep.ActionHalt
-	}
-
-	return multistep.ActionContinue
-}
-
-func (*stepProvision) Cleanup(map[string]interface{}) {}

builder/common/download.go

@@ -2,6 +2,9 @@ package common
 
 import (
 	"bytes"
+	"crypto/md5"
+	"crypto/sha1"
+	"crypto/sha256"
 	"encoding/hex"
 	"errors"
 	"fmt"
@@ -46,6 +49,21 @@ type DownloadClient struct {
 	downloader Downloader
 }
 
+// HashForType returns the Hash implementation for the given string
+// type, or nil if the type is not supported.
+func HashForType(t string) hash.Hash {
+	switch t {
+	case "md5":
+		return md5.New()
+	case "sha1":
+		return sha1.New()
+	case "sha256":
+		return sha256.New()
+	default:
+		return nil
+	}
+}
+
 // NewDownloadClient returns a new DownloadClient for the given
 // configuration.
 func NewDownloadClient(c *DownloadConfig) *DownloadClient {
@@ -107,6 +125,9 @@ func (d *DownloadClient) Get() (string, error) {
 
 		log.Printf("Downloading: %s", url.String())
 		err = d.downloader.Download(f, url)
+		if err != nil {
+			return "", err
+		}
 	}
 
 	if d.config.Hash != nil {
@@ -160,11 +181,22 @@ func (*HTTPDownloader) Cancel() {
 }
 
 func (d *HTTPDownloader) Download(dst io.Writer, src *url.URL) error {
+	log.Printf("Starting download: %s", src.String())
 	resp, err := http.Get(src.String())
 	if err != nil {
 		return err
 	}
 
+	if resp.StatusCode != 200 {
+		log.Printf(
+			"Non-200 status code: %d. Getting error body.", resp.StatusCode)
+
+		errorBody := new(bytes.Buffer)
+		io.Copy(errorBody, resp.Body)
+		return fmt.Errorf("HTTP error '%d'! Remote side responded:\n%s",
+			resp.StatusCode, errorBody.String())
+	}
+
 	d.progress = 0
 	d.total = uint(resp.ContentLength)
 

builder/common/download_test.go

@@ -40,3 +40,48 @@ func TestDownloadClient_VerifyChecksum(t *testing.T) {
 		t.Fatal("didn't verify")
 	}
 }
+
+func TestHashForType(t *testing.T) {
+	if h := HashForType("md5"); h == nil {
+		t.Fatalf("md5 hash is nil")
+	} else {
+		h.Write([]byte("foo"))
+		result := h.Sum(nil)
+
+		expected := "acbd18db4cc2f85cedef654fccc4a4d8"
+		actual := hex.EncodeToString(result)
+		if actual != expected {
+			t.Fatalf("bad hash: %s", actual)
+		}
+	}
+
+	if h := HashForType("sha1"); h == nil {
+		t.Fatalf("sha1 hash is nil")
+	} else {
+		h.Write([]byte("foo"))
+		result := h.Sum(nil)
+
+		expected := "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"
+		actual := hex.EncodeToString(result)
+		if actual != expected {
+			t.Fatalf("bad hash: %s", actual)
+		}
+	}
+
+	if h := HashForType("sha256"); h == nil {
+		t.Fatalf("sha256 hash is nil")
+	} else {
+		h.Write([]byte("foo"))
+		result := h.Sum(nil)
+
+		expected := "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"
+		actual := hex.EncodeToString(result)
+		if actual != expected {
+			t.Fatalf("bad hash: %s", actual)
+		}
+	}
+
+	if HashForType("fake") != nil {
+		t.Fatalf("fake hash is not nil")
+	}
+}

builder/common/step_connect_ssh.go

@@ -0,0 +1,157 @@
+package common
+
+import (
+	gossh "code.google.com/p/go.crypto/ssh"
+	"errors"
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/communicator/ssh"
+	"github.com/mitchellh/packer/packer"
+	"log"
+	"strings"
+	"time"
+)
+
+// StepConnectSSH is a multistep Step implementation that waits for SSH
+// to become available. It gets the connection information from a single
+// configuration when creating the step.
+//
+// Uses:
+//   ui packer.Ui
+//
+// Produces:
+//   communicator packer.Communicator
+type StepConnectSSH struct {
+	// SSHAddress is a function that returns the TCP address to connect to
+	// for SSH. This is a function so that you can query information
+	// if necessary for this address.
+	SSHAddress func(map[string]interface{}) (string, error)
+
+	// SSHConfig is a function that returns the proper client configuration
+	// for SSH access.
+	SSHConfig func(map[string]interface{}) (*gossh.ClientConfig, error)
+
+	// SSHWaitTimeout is the total timeout to wait for SSH to become available.
+	SSHWaitTimeout time.Duration
+
+	cancel bool
+	comm   packer.Communicator
+}
+
+func (s *StepConnectSSH) Run(state map[string]interface{}) multistep.StepAction {
+	ui := state["ui"].(packer.Ui)
+
+	var comm packer.Communicator
+	var err error
+
+	waitDone := make(chan bool, 1)
+	go func() {
+		ui.Say("Waiting for SSH to become available...")
+		comm, err = s.waitForSSH(state)
+		waitDone <- true
+	}()
+
+	log.Printf("Waiting for SSH, up to timeout: %s", s.SSHWaitTimeout)
+	timeout := time.After(s.SSHWaitTimeout)
+WaitLoop:
+	for {
+		// Wait for either SSH to become available, a timeout to occur,
+		// or an interrupt to come through.
+		select {
+		case <-waitDone:
+			if err != nil {
+				ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
+				return multistep.ActionHalt
+			}
+
+			ui.Say("Connected to SSH!")
+			s.comm = comm
+			state["communicator"] = comm
+			break WaitLoop
+		case <-timeout:
+			ui.Error("Timeout waiting for SSH.")
+			s.cancel = true
+			return multistep.ActionHalt
+		case <-time.After(1 * time.Second):
+			if _, ok := state[multistep.StateCancelled]; ok {
+				// The step sequence was cancelled, so cancel waiting for SSH
+				// and just start the halting process.
+				s.cancel = true
+				log.Println("Interrupt detected, quitting waiting for SSH.")
+				return multistep.ActionHalt
+			}
+		}
+	}
+
+	return multistep.ActionContinue
+}
+
+func (s *StepConnectSSH) Cleanup(map[string]interface{}) {
+}
+
+func (s *StepConnectSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
+	handshakeAttempts := 0
+
+	var comm packer.Communicator
+	for {
+		time.Sleep(5 * time.Second)
+
+		if s.cancel {
+			log.Println("SSH wait cancelled. Exiting loop.")
+			return nil, errors.New("SSH wait cancelled")
+		}
+
+		// First we request the TCP connection information
+		address, err := s.SSHAddress(state)
+		if err != nil {
+			log.Printf("Error getting SSH address: %s", err)
+			continue
+		}
+
+		// Retrieve the SSH configuration
+		sshConfig, err := s.SSHConfig(state)
+		if err != nil {
+			log.Printf("Error getting SSH config: %s", err)
+			continue
+		}
+
+		// Attempt to connect to SSH port
+		connFunc := ssh.ConnectFunc("tcp", address, 5*time.Minute)
+		nc, err := connFunc()
+		if err != nil {
+			log.Printf("TCP connection to SSH ip/port failed: %s", err)
+			continue
+		}
+		nc.Close()
+
+		// Then we attempt to connect via SSH
+		config := &ssh.Config{
+			Connection: connFunc,
+			SSHConfig:  sshConfig,
+		}
+
+		comm, err = ssh.New(config)
+		if err != nil {
+			log.Printf("SSH handshake err: %s", err)
+
+			// Only count this as an attempt if we were able to attempt
+			// to authenticate. Note this is very brittle since it depends
+			// on the string of the error... but I don't see any other way.
+			if strings.Contains(err.Error(), "authenticate") {
+				log.Printf("Detected authentication error. Increasing handshake attempts.")
+				handshakeAttempts += 1
+			}
+
+			if handshakeAttempts < 10 {
+				// Try to connect via SSH a handful of times
+				continue
+			}
+
+			return nil, err
+		}
+
+		break
+	}
+
+	return comm, nil
+}

builder/common/step_connect_ssh_test.go

@@ -0,0 +1,14 @@
+package common
+
+import (
+	"github.com/mitchellh/multistep"
+	"testing"
+)
+
+func TestStepConnectSSH_Impl(t *testing.T) {
+	var raw interface{}
+	raw = new(StepConnectSSH)
+	if _, ok := raw.(multistep.Step); !ok {
+		t.Fatalf("connect ssh should be a step")
+	}
+}

builder/common/step_create_floppy.go

@@ -0,0 +1,135 @@
+package common
+
+import (
+	"fmt"
+	"github.com/mitchellh/go-fs"
+	"github.com/mitchellh/go-fs/fat"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/packer"
+	"io"
+	"io/ioutil"
+	"log"
+	"os"
+	"path/filepath"
+)
+
+// StepCreateFloppy will create a floppy disk with the given files.
+// The floppy disk doesn't support sub-directories. Only files at the
+// root level are supported.
+type StepCreateFloppy struct {
+	Files []string
+
+	floppyPath string
+}
+
+func (s *StepCreateFloppy) Run(state map[string]interface{}) multistep.StepAction {
+	if len(s.Files) == 0 {
+		log.Println("No floppy files specified. Floppy disk will not be made.")
+		return multistep.ActionContinue
+	}
+
+	ui := state["ui"].(packer.Ui)
+	ui.Say("Creating floppy disk...")
+
+	// Create a temporary file to be our floppy drive
+	floppyF, err := ioutil.TempFile("", "packer")
+	if err != nil {
+		state["error"] = fmt.Errorf("Error creating temporary file for floppy: %s", err)
+		return multistep.ActionHalt
+	}
+	defer floppyF.Close()
+
+	// Set the path so we can remove it later
+	s.floppyPath = floppyF.Name()
+
+	log.Printf("Floppy path: %s", floppyF.Name())
+
+	// Set the size of the file to be a floppy sized
+	if err := floppyF.Truncate(1440 * 1024); err != nil {
+		state["error"] = fmt.Errorf("Error creating floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// BlockDevice backed by the file for our filesystem
+	log.Println("Initializing block device backed by temporary file")
+	device, err := fs.NewFileDisk(floppyF)
+	if err != nil {
+		state["error"] = fmt.Errorf("Error creating floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// Format the block device so it contains a valid FAT filesystem
+	log.Println("Formatting the block device with a FAT filesystem...")
+	formatConfig := &fat.SuperFloppyConfig{
+		FATType: fat.FAT12,
+		Label:   "packer",
+		OEMName: "packer",
+	}
+	if fat.FormatSuperFloppy(device, formatConfig); err != nil {
+		state["error"] = fmt.Errorf("Error creating floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// The actual FAT filesystem
+	log.Println("Initializing FAT filesystem on block device")
+	fatFs, err := fat.New(device)
+	if err != nil {
+		state["error"] = fmt.Errorf("Error creating floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// Get the root directory to the filesystem
+	log.Println("Reading the root directory from the filesystem")
+	rootDir, err := fatFs.RootDir()
+	if err != nil {
+		state["error"] = fmt.Errorf("Error creating floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// Go over each file and copy it.
+	for _, filename := range s.Files {
+		ui.Message(fmt.Sprintf("Copying: %s", filepath.Base(filename)))
+		if s.addSingleFile(rootDir, filename); err != nil {
+			state["error"] = fmt.Errorf("Error adding file to floppy: %s", err)
+			return multistep.ActionHalt
+		}
+	}
+
+	// Set the path to the floppy so it can be used later
+	state["floppy_path"] = s.floppyPath
+
+	return multistep.ActionContinue
+}
+
+func (s *StepCreateFloppy) Cleanup(map[string]interface{}) {
+	if s.floppyPath != "" {
+		log.Printf("Deleting floppy disk: %s", s.floppyPath)
+		os.Remove(s.floppyPath)
+	}
+}
+
+func (s *StepCreateFloppy) addSingleFile(dir fs.Directory, src string) error {
+	log.Printf("Adding file to floppy: %s", src)
+
+	inputF, err := os.Open(src)
+	if err != nil {
+		return err
+	}
+	defer inputF.Close()
+
+	entry, err := dir.AddFile(filepath.Base(src))
+	if err != nil {
+		return err
+	}
+
+	fatFile, err := entry.File()
+	if err != nil {
+		return err
+	}
+
+	if _, err := io.Copy(fatFile, inputF); err != nil {
+		return err
+	}
+
+	return nil
+}

builder/common/step_provision.go

@@ -1,4 +1,4 @@
-package digitalocean
+package common
 
 import (
 	"github.com/mitchellh/multistep"
@@ -6,9 +6,18 @@ import (
 	"log"
 )
 
-type stepProvision struct{}
+// StepProvision runs the provisioners.
+//
+// Uses:
+//   communicator packer.Communicator
+//   hook         packer.Hook
+//   ui           packer.Ui
+//
+// Produces:
+//   <nothing>
+type StepProvision struct{}
 
-func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
+func (*StepProvision) Run(state map[string]interface{}) multistep.StepAction {
 	comm := state["communicator"].(packer.Communicator)
 	hook := state["hook"].(packer.Hook)
 	ui := state["ui"].(packer.Ui)
@@ -22,4 +31,4 @@ func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
 	return multistep.ActionContinue
 }
 
-func (*stepProvision) Cleanup(map[string]interface{}) {}
+func (*StepProvision) Cleanup(map[string]interface{}) {}

builder/common/step_provision_test.go

@@ -0,0 +1,14 @@
+package common
+
+import (
+	"github.com/mitchellh/multistep"
+	"testing"
+)
+
+func TestStepProvision_Impl(t *testing.T) {
+	var raw interface{}
+	raw = new(StepProvision)
+	if _, ok := raw.(multistep.Step); !ok {
+		t.Fatalf("provision should be a step")
+	}
+}

builder/digitalocean/api.go

@@ -189,28 +189,25 @@ func NewRequest(d DigitalOceanClient, path string, params string) (map[string]in
 		return decodedResponse, err
 	}
 
+	log.Printf("response from digitalocean: %s", body)
+
 	err = json.Unmarshal(body, &decodedResponse)
 
-	log.Printf("response from digitalocean: %v", decodedResponse)
-
-	// Catch all non-200 status and return an error
-	if resp.StatusCode != 200 {
-		err = errors.New(fmt.Sprintf("Received non-200 HTTP status from DigitalOcean: %v", resp.StatusCode))
-		return decodedResponse, err
-	}
-
+	// Check for bad JSON
 	if err != nil {
+		err = errors.New(fmt.Sprintf("Failed to decode JSON response (HTTP %v) from DigitalOcean: %s",
+			resp.StatusCode, body))
 		return decodedResponse, err
 	}
 
-	// Catch all non-OK statuses from DO and return an error
+	// Check for errors sent by digitalocean
 	status := decodedResponse["status"]
 	if status != "OK" {
 		// Get the actual error message if there is one
 		if status == "ERROR" {
 			status = decodedResponse["error_message"]
 		}
-		err = errors.New(fmt.Sprintf("Received bad status from DigitalOcean: %v", status))
+		err = errors.New(fmt.Sprintf("Received bad response (HTTP %v) from DigitalOcean: %s", resp.StatusCode, status))
 		return decodedResponse, err
 	}
 

builder/digitalocean/builder.go

@@ -12,7 +12,10 @@ import (
 	"github.com/mitchellh/packer/builder/common"
 	"github.com/mitchellh/packer/packer"
 	"log"
+	"os"
+	"sort"
 	"strconv"
+	"strings"
 	"text/template"
 	"time"
 )
@@ -37,9 +40,6 @@ type config struct {
 	SnapshotName string
 	SSHUsername  string `mapstructure:"ssh_username"`
 	SSHPort      uint   `mapstructure:"ssh_port"`
-	SSHTimeout   time.Duration
-	EventDelay   time.Duration
-	StateTimeout time.Duration
 
 	PackerDebug bool `mapstructure:"packer_debug"`
 
@@ -47,6 +47,12 @@ type config struct {
 	RawSSHTimeout   string `mapstructure:"ssh_timeout"`
 	RawEventDelay   string `mapstructure:"event_delay"`
 	RawStateTimeout string `mapstructure:"state_timeout"`
+
+	// These are unexported since they're set by other fields
+	// being set.
+	sshTimeout   time.Duration
+	eventDelay   time.Duration
+	stateTimeout time.Duration
 }
 
 type Builder struct {
@@ -55,15 +61,49 @@ type Builder struct {
 }
 
 func (b *Builder) Prepare(raws ...interface{}) error {
+	var md mapstructure.Metadata
+	decoderConfig := &mapstructure.DecoderConfig{
+		Metadata: &md,
+		Result:   &b.config,
+	}
+
+	decoder, err := mapstructure.NewDecoder(decoderConfig)
+	if err != nil {
+		return err
+	}
+
 	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &b.config)
+		err := decoder.Decode(raw)
 		if err != nil {
 			return err
 		}
 	}
 
+	// Accumulate any errors
+	errs := make([]error, 0)
+
+	// Unused keys are errors
+	if len(md.Unused) > 0 {
+		sort.Strings(md.Unused)
+		for _, unused := range md.Unused {
+			if unused != "type" && !strings.HasPrefix(unused, "packer_") {
+				errs = append(
+					errs, fmt.Errorf("Unknown configuration key: %s", unused))
+			}
+		}
+	}
+
 	// Optional configuration with defaults
-	//
+	if b.config.APIKey == "" {
+		// Default to environment variable for api_key, if it exists
+		b.config.APIKey = os.Getenv("DIGITALOCEAN_API_KEY")
+	}
+
+	if b.config.ClientID == "" {
+		// Default to environment variable for client_id, if it exists
+		b.config.ClientID = os.Getenv("DIGITALOCEAN_CLIENT_ID")
+	}
+
 	if b.config.RegionID == 0 {
 		// Default to Region "New York"
 		b.config.RegionID = 1
@@ -112,11 +152,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.RawStateTimeout = "6m"
 	}
 
-	// A list of errors on the configuration
-	errs := make([]error, 0)
-
 	// Required configurations that will display errors if not set
-	//
 	if b.config.ClientID == "" {
 		errs = append(errs, errors.New("a client_id must be specified"))
 	}
@@ -129,19 +165,19 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing ssh_timeout: %s", err))
 	}
-	b.config.SSHTimeout = sshTimeout
+	b.config.sshTimeout = sshTimeout
 
 	eventDelay, err := time.ParseDuration(b.config.RawEventDelay)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing event_delay: %s", err))
 	}
-	b.config.EventDelay = eventDelay
+	b.config.eventDelay = eventDelay
 
 	stateTimeout, err := time.ParseDuration(b.config.RawStateTimeout)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing state_timeout: %s", err))
 	}
-	b.config.StateTimeout = stateTimeout
+	b.config.stateTimeout = stateTimeout
 
 	// Parse the name of the snapshot
 	snapNameBuf := new(bytes.Buffer)
@@ -180,8 +216,12 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		new(stepCreateSSHKey),
 		new(stepCreateDroplet),
 		new(stepDropletInfo),
-		new(stepConnectSSH),
-		new(stepProvision),
+		&common.StepConnectSSH{
+			SSHAddress:     sshAddress,
+			SSHConfig:      sshConfig,
+			SSHWaitTimeout: 5 * time.Minute,
+		},
+		new(common.StepProvision),
 		new(stepPowerOff),
 		new(stepSnapshot),
 	}

builder/digitalocean/builder_test.go

@@ -2,10 +2,17 @@ package digitalocean
 
 import (
 	"github.com/mitchellh/packer/packer"
+	"os"
 	"strconv"
 	"testing"
 )
 
+func init() {
+	// Clear out the credential env vars
+	os.Setenv("DIGITALOCEAN_API_KEY", "")
+	os.Setenv("DIGITALOCEAN_CLIENT_ID", "")
+}
+
 func testConfig() map[string]interface{} {
 	return map[string]interface{}{
 		"client_id": "foo",
@@ -55,6 +62,15 @@ func TestBuilderPrepare_APIKey(t *testing.T) {
 	if err == nil {
 		t.Fatal("should have error")
 	}
+
+	// Test env variable
+	delete(config, "api_key")
+	os.Setenv("DIGITALOCEAN_API_KEY", "foo")
+	defer os.Setenv("DIGITALOCEAN_API_KEY", "")
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
 }
 
 func TestBuilderPrepare_ClientID(t *testing.T) {
@@ -79,6 +95,27 @@ func TestBuilderPrepare_ClientID(t *testing.T) {
 	if err == nil {
 		t.Fatal("should have error")
 	}
+
+	// Test env variable
+	delete(config, "client_id")
+	os.Setenv("DIGITALOCEAN_CLIENT_ID", "foo")
+	defer os.Setenv("DIGITALOCEAN_CLIENT_ID", "")
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+}
+
+func TestBuilderPrepare_InvalidKey(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Add a random key
+	config["i_should_not_be_valid"] = true
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
 }
 
 func TestBuilderPrepare_RegionID(t *testing.T) {

builder/digitalocean/ssh.go

@@ -0,0 +1,30 @@
+package digitalocean
+
+import (
+	gossh "code.google.com/p/go.crypto/ssh"
+	"fmt"
+	"github.com/mitchellh/packer/communicator/ssh"
+)
+
+func sshAddress(state map[string]interface{}) (string, error) {
+	config := state["config"].(config)
+	ipAddress := state["droplet_ip"].(string)
+	return fmt.Sprintf("%s:%d", ipAddress, config.SSHPort), nil
+}
+
+func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
+	config := state["config"].(config)
+	privateKey := state["privateKey"].(string)
+
+	keyring := new(ssh.SimpleKeychain)
+	if err := keyring.AddPEMKey(privateKey); err != nil {
+		return nil, fmt.Errorf("Error setting up SSH config: %s", err)
+	}
+
+	return &gossh.ClientConfig{
+		User: config.SSHUsername,
+		Auth: []gossh.ClientAuth{
+			gossh.ClientAuthKeyring(keyring),
+		},
+	}, nil
+}

builder/digitalocean/step_connect_ssh.go

@@ -1,137 +0,0 @@
-package digitalocean
-
-import (
-	gossh "code.google.com/p/go.crypto/ssh"
-	"errors"
-	"fmt"
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/communicator/ssh"
-	"github.com/mitchellh/packer/packer"
-	"log"
-	"net"
-	"time"
-)
-
-type stepConnectSSH struct {
-	conn net.Conn
-}
-
-func (s *stepConnectSSH) Run(state map[string]interface{}) multistep.StepAction {
-	config := state["config"].(config)
-	privateKey := state["privateKey"].(string)
-	ui := state["ui"].(packer.Ui)
-	ipAddress := state["droplet_ip"]
-
-	// Build the keyring for authentication. This stores the private key
-	// we'll use to authenticate.
-	keyring := &ssh.SimpleKeychain{}
-	err := keyring.AddPEMKey(privateKey)
-	if err != nil {
-		err := fmt.Errorf("Error setting up SSH config: %s", err)
-		state["error"] = err
-		ui.Error(err.Error())
-		return multistep.ActionHalt
-	}
-
-	// Build the actual SSH client configuration
-	sshConfig := &gossh.ClientConfig{
-		User: config.SSHUsername,
-		Auth: []gossh.ClientAuth{
-			gossh.ClientAuthKeyring(keyring),
-		},
-	}
-
-	// Start trying to connect to SSH
-	connected := make(chan error, 1)
-	connectQuit := make(chan bool, 1)
-	defer func() {
-		connectQuit <- true
-	}()
-
-	var comm packer.Communicator
-	go func() {
-		var err error
-
-		ui.Say("Connecting to the droplet via SSH...")
-		attempts := 0
-		handshakeAttempts := 0
-		for {
-			select {
-			case <-connectQuit:
-				return
-			default:
-			}
-
-			attempts += 1
-			log.Printf(
-				"Opening TCP conn for SSH to %s:%d (attempt %d)",
-				ipAddress, config.SSHPort, attempts)
-			s.conn, err = net.DialTimeout(
-				"tcp",
-				fmt.Sprintf("%s:%d", ipAddress, config.SSHPort),
-				10*time.Second)
-			if err == nil {
-				log.Println("TCP connection made. Attempting SSH handshake.")
-				comm, err = ssh.New(s.conn, sshConfig)
-				if err == nil {
-					log.Println("Connected to SSH!")
-					break
-				}
-
-				handshakeAttempts += 1
-				log.Printf("SSH handshake error: %s", err)
-
-				if handshakeAttempts > 5 {
-					connected <- err
-					return
-				}
-			}
-
-			// A brief sleep so we're not being overly zealous attempting
-			// to connect to the instance.
-			time.Sleep(500 * time.Millisecond)
-		}
-
-		connected <- nil
-	}()
-
-	log.Printf("Waiting up to %s for SSH connection", config.SSHTimeout)
-	timeout := time.After(config.SSHTimeout)
-
-ConnectWaitLoop:
-	for {
-		select {
-		case err := <-connected:
-			if err != nil {
-				err := fmt.Errorf("Error connecting to SSH: %s", err)
-				state["error"] = err
-				ui.Error(err.Error())
-				return multistep.ActionHalt
-			}
-
-			// We connected. Just break the loop.
-			break ConnectWaitLoop
-		case <-timeout:
-			err := errors.New("Timeout waiting for SSH to become available.")
-			state["error"] = err
-			ui.Error(err.Error())
-			return multistep.ActionHalt
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				log.Println("Interrupt detected, quitting waiting for SSH.")
-				return multistep.ActionHalt
-			}
-		}
-	}
-
-	// Set the communicator on the state bag so it can be used later
-	state["communicator"] = comm
-
-	return multistep.ActionContinue
-}
-
-func (s *stepConnectSSH) Cleanup(map[string]interface{}) {
-	if s.conn != nil {
-		s.conn.Close()
-	}
-}

builder/digitalocean/step_create_droplet.go

@@ -60,7 +60,7 @@ func (s *stepCreateDroplet) Cleanup(state map[string]interface{}) {
 	// Otherwise we get "pending event" errors, even though there isn't
 	// one.
 	log.Printf("Sleeping for %v, event_delay", c.RawEventDelay)
-	time.Sleep(c.EventDelay)
+	time.Sleep(c.eventDelay)
 
 	err := client.DestroyDroplet(s.dropletId)
 

builder/digitalocean/step_power_off.go

@@ -20,7 +20,7 @@ func (s *stepPowerOff) Run(state map[string]interface{}) multistep.StepAction {
 	// Otherwise we get "pending event" errors, even though there isn't
 	// one.
 	log.Printf("Sleeping for %v, event_delay", c.RawEventDelay)
-	time.Sleep(c.EventDelay)
+	time.Sleep(c.eventDelay)
 
 	// Poweroff the droplet so it can be snapshot
 	err := client.PowerOffDroplet(dropletId)

builder/digitalocean/wait.go

@@ -17,9 +17,7 @@ func waitForDropletState(desiredState string, dropletId uint, client *DigitalOce
 			attempts += 1
 
 			log.Printf("Checking droplet status... (attempt: %d)", attempts)
-
 			_, status, err := client.DropletStatus(dropletId)
-
 			if err != nil {
 				log.Println(err)
 				break
@@ -37,7 +35,7 @@ func waitForDropletState(desiredState string, dropletId uint, client *DigitalOce
 	}()
 
 	log.Printf("Waiting for up to %s for droplet to become %s", c.RawStateTimeout, desiredState)
-	timeout := time.After(c.StateTimeout)
+	timeout := time.After(c.stateTimeout)
 
 ActiveWaitLoop:
 	for {

builder/virtualbox/builder.go

@@ -12,6 +12,7 @@ import (
 	"os"
 	"os/exec"
 	"path/filepath"
+	"sort"
 	"strings"
 	"time"
 )
@@ -25,50 +26,85 @@ type Builder struct {
 }
 
 type config struct {
-	BootCommand        []string      `mapstructure:"boot_command"`
-	BootWait           time.Duration ``
-	DiskSize           uint          `mapstructure:"disk_size"`
-	GuestAdditionsPath string        `mapstructure:"guest_additions_path"`
-	GuestOSType        string        `mapstructure:"guest_os_type"`
-	HTTPDir            string        `mapstructure:"http_directory"`
-	HTTPPortMin        uint          `mapstructure:"http_port_min"`
-	HTTPPortMax        uint          `mapstructure:"http_port_max"`
-	ISOMD5             string        `mapstructure:"iso_md5"`
-	ISOUrl             string        `mapstructure:"iso_url"`
-	OutputDir          string        `mapstructure:"output_directory"`
-	ShutdownCommand    string        `mapstructure:"shutdown_command"`
-	ShutdownTimeout    time.Duration ``
-	SSHHostPortMin     uint          `mapstructure:"ssh_host_port_min"`
-	SSHHostPortMax     uint          `mapstructure:"ssh_host_port_max"`
-	SSHPassword        string        `mapstructure:"ssh_password"`
-	SSHPort            uint          `mapstructure:"ssh_port"`
-	SSHUser            string        `mapstructure:"ssh_username"`
-	SSHWaitTimeout     time.Duration ``
-	VBoxVersionFile    string        `mapstructure:"virtualbox_version_file"`
-	VBoxManage         [][]string    `mapstructure:"vboxmanage"`
-	VMName             string        `mapstructure:"vm_name"`
+	BootCommand          []string   `mapstructure:"boot_command"`
+	DiskSize             uint       `mapstructure:"disk_size"`
+	FloppyFiles          []string   `mapstructure:"floppy_files"`
+	GuestAdditionsPath   string     `mapstructure:"guest_additions_path"`
+	GuestAdditionsURL    string     `mapstructure:"guest_additions_url"`
+	GuestAdditionsSHA256 string     `mapstructure:"guest_additions_sha256"`
+	GuestOSType          string     `mapstructure:"guest_os_type"`
+	Headless             bool       `mapstructure:"headless"`
+	HTTPDir              string     `mapstructure:"http_directory"`
+	HTTPPortMin          uint       `mapstructure:"http_port_min"`
+	HTTPPortMax          uint       `mapstructure:"http_port_max"`
+	ISOChecksum          string     `mapstructure:"iso_checksum"`
+	ISOChecksumType      string     `mapstructure:"iso_checksum_type"`
+	ISOUrl               string     `mapstructure:"iso_url"`
+	OutputDir            string     `mapstructure:"output_directory"`
+	ShutdownCommand      string     `mapstructure:"shutdown_command"`
+	SSHHostPortMin       uint       `mapstructure:"ssh_host_port_min"`
+	SSHHostPortMax       uint       `mapstructure:"ssh_host_port_max"`
+	SSHPassword          string     `mapstructure:"ssh_password"`
+	SSHPort              uint       `mapstructure:"ssh_port"`
+	SSHUser              string     `mapstructure:"ssh_username"`
+	VBoxVersionFile      string     `mapstructure:"virtualbox_version_file"`
+	VBoxManage           [][]string `mapstructure:"vboxmanage"`
+	VMName               string     `mapstructure:"vm_name"`
 
-	PackerDebug bool `mapstructure:"packer_debug"`
+	PackerBuildName string `mapstructure:"packer_build_name"`
+	PackerDebug     bool   `mapstructure:"packer_debug"`
+	PackerForce     bool   `mapstructure:"packer_force"`
 
 	RawBootWait        string `mapstructure:"boot_wait"`
 	RawShutdownTimeout string `mapstructure:"shutdown_timeout"`
 	RawSSHWaitTimeout  string `mapstructure:"ssh_wait_timeout"`
+
+	bootWait        time.Duration ``
+	shutdownTimeout time.Duration ``
+	sshWaitTimeout  time.Duration ``
 }
 
 func (b *Builder) Prepare(raws ...interface{}) error {
-	var err error
+	var md mapstructure.Metadata
+	decoderConfig := &mapstructure.DecoderConfig{
+		Metadata: &md,
+		Result:   &b.config,
+	}
+
+	decoder, err := mapstructure.NewDecoder(decoderConfig)
+	if err != nil {
+		return err
+	}
 
 	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &b.config)
+		err := decoder.Decode(raw)
 		if err != nil {
 			return err
 		}
 	}
 
+	// Accumulate any errors
+	errs := make([]error, 0)
+
+	// Unused keys are errors
+	if len(md.Unused) > 0 {
+		sort.Strings(md.Unused)
+		for _, unused := range md.Unused {
+			if unused != "type" && !strings.HasPrefix(unused, "packer_") {
+				errs = append(
+					errs, fmt.Errorf("Unknown configuration key: %s", unused))
+			}
+		}
+	}
+
 	if b.config.DiskSize == 0 {
 		b.config.DiskSize = 40000
 	}
 
+	if b.config.FloppyFiles == nil {
+		b.config.FloppyFiles = make([]string, 0)
+	}
+
 	if b.config.GuestAdditionsPath == "" {
 		b.config.GuestAdditionsPath = "VBoxGuestAdditions.iso"
 	}
@@ -86,7 +122,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 	}
 
 	if b.config.OutputDir == "" {
-		b.config.OutputDir = "virtualbox"
+		b.config.OutputDir = fmt.Sprintf("output-%s", b.config.PackerBuildName)
 	}
 
 	if b.config.RawBootWait == "" {
@@ -114,19 +150,28 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 	}
 
 	if b.config.VMName == "" {
-		b.config.VMName = "packer"
+		b.config.VMName = fmt.Sprintf("packer-%s", b.config.PackerBuildName)
 	}
 
-	errs := make([]error, 0)
-
 	if b.config.HTTPPortMin > b.config.HTTPPortMax {
 		errs = append(errs, errors.New("http_port_min must be less than http_port_max"))
 	}
 
-	if b.config.ISOMD5 == "" {
-		errs = append(errs, errors.New("Due to large file sizes, an iso_md5 is required"))
+	if b.config.ISOChecksum == "" {
+		errs = append(errs, errors.New("Due to large file sizes, an iso_checksum is required"))
 	} else {
-		b.config.ISOMD5 = strings.ToLower(b.config.ISOMD5)
+		b.config.ISOChecksum = strings.ToLower(b.config.ISOChecksum)
+	}
+
+	if b.config.ISOChecksumType == "" {
+		errs = append(errs, errors.New("The iso_checksum_type must be specified."))
+	} else {
+		b.config.ISOChecksumType = strings.ToLower(b.config.ISOChecksumType)
+		if h := common.HashForType(b.config.ISOChecksumType); h == nil {
+			errs = append(
+				errs,
+				fmt.Errorf("Unsupported checksum type: %s", b.config.ISOChecksumType))
+		}
 	}
 
 	if b.config.ISOUrl == "" {
@@ -168,11 +213,56 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
-	if _, err := os.Stat(b.config.OutputDir); err == nil {
-		errs = append(errs, errors.New("Output directory already exists. It must not exist."))
+	if b.config.GuestAdditionsSHA256 != "" {
+		b.config.GuestAdditionsSHA256 = strings.ToLower(b.config.GuestAdditionsSHA256)
 	}
 
-	b.config.BootWait, err = time.ParseDuration(b.config.RawBootWait)
+	if b.config.GuestAdditionsURL != "" {
+		url, err := url.Parse(b.config.GuestAdditionsURL)
+		if err != nil {
+			errs = append(errs, fmt.Errorf("guest_additions_url is not a valid URL: %s", err))
+		} else {
+			if url.Scheme == "" {
+				url.Scheme = "file"
+			}
+
+			if url.Scheme == "file" {
+				if _, err := os.Stat(url.Path); err != nil {
+					errs = append(errs, fmt.Errorf("guest_additions_url points to bad file: %s", err))
+				}
+			} else {
+				supportedSchemes := []string{"file", "http", "https"}
+				scheme := strings.ToLower(url.Scheme)
+
+				found := false
+				for _, supported := range supportedSchemes {
+					if scheme == supported {
+						found = true
+						break
+					}
+				}
+
+				if !found {
+					errs = append(errs, fmt.Errorf("Unsupported URL scheme in guest_additions_url: %s", scheme))
+				}
+			}
+		}
+
+		if len(errs) == 0 {
+			// Put the URL back together since we may have modified it
+			b.config.GuestAdditionsURL = url.String()
+		}
+	}
+
+	if !b.config.PackerForce {
+		if _, err := os.Stat(b.config.OutputDir); err == nil {
+			errs = append(
+				errs,
+				errors.New("Output directory already exists. It must not exist."))
+		}
+	}
+
+	b.config.bootWait, err = time.ParseDuration(b.config.RawBootWait)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing boot_wait: %s", err))
 	}
@@ -185,7 +275,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.RawSSHWaitTimeout = "20m"
 	}
 
-	b.config.ShutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
+	b.config.shutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing shutdown_timeout: %s", err))
 	}
@@ -198,7 +288,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		errs = append(errs, errors.New("An ssh_username must be specified."))
 	}
 
-	b.config.SSHWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
+	b.config.sshWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing ssh_wait_timeout: %s", err))
 	}
@@ -220,19 +310,27 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		new(stepDownloadGuestAdditions),
 		new(stepDownloadISO),
 		new(stepPrepareOutputDir),
+		&common.StepCreateFloppy{
+			Files: b.config.FloppyFiles,
+		},
 		new(stepHTTPServer),
 		new(stepSuppressMessages),
 		new(stepCreateVM),
 		new(stepCreateDisk),
 		new(stepAttachISO),
+		new(stepAttachFloppy),
 		new(stepForwardSSH),
 		new(stepVBoxManage),
 		new(stepRun),
 		new(stepTypeBootCommand),
-		new(stepWaitForSSH),
+		&common.StepConnectSSH{
+			SSHAddress:     sshAddress,
+			SSHConfig:      sshConfig,
+			SSHWaitTimeout: b.config.sshWaitTimeout,
+		},
 		new(stepUploadVersion),
 		new(stepUploadGuestAdditions),
-		new(stepProvision),
+		new(common.StepProvision),
 		new(stepShutdown),
 		new(stepExport),
 	}

builder/virtualbox/builder_test.go

@@ -10,9 +10,12 @@ import (
 
 func testConfig() map[string]interface{} {
 	return map[string]interface{}{
-		"iso_md5":      "foo",
-		"iso_url":      "http://www.google.com/",
-		"ssh_username": "foo",
+		"iso_checksum":      "foo",
+		"iso_checksum_type": "md5",
+		"iso_url":           "http://www.google.com/",
+		"ssh_username":      "foo",
+
+		packer.BuildNameConfigKey: "foo",
 	}
 }
 
@@ -36,7 +39,7 @@ func TestBuilderPrepare_Defaults(t *testing.T) {
 		t.Errorf("bad guest OS type: %s", b.config.GuestOSType)
 	}
 
-	if b.config.OutputDir != "virtualbox" {
+	if b.config.OutputDir != "output-foo" {
 		t.Errorf("bad output dir: %s", b.config.OutputDir)
 	}
 
@@ -52,7 +55,7 @@ func TestBuilderPrepare_Defaults(t *testing.T) {
 		t.Errorf("bad ssh port: %d", b.config.SSHPort)
 	}
 
-	if b.config.VMName != "packer" {
+	if b.config.VMName != "packer-foo" {
 		t.Errorf("bad vm name: %s", b.config.VMName)
 	}
 }
@@ -113,11 +116,38 @@ func TestBuilderPrepare_DiskSize(t *testing.T) {
 	}
 }
 
+func TestBuilderPrepare_FloppyFiles(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	delete(config, "floppy_files")
+	err := b.Prepare(config)
+	if err != nil {
+		t.Fatalf("bad err: %s", err)
+	}
+
+	if len(b.config.FloppyFiles) != 0 {
+		t.Fatalf("bad: %#v", b.config.FloppyFiles)
+	}
+
+	config["floppy_files"] = []string{"foo", "bar"}
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	expected := []string{"foo", "bar"}
+	if !reflect.DeepEqual(b.config.FloppyFiles, expected) {
+		t.Fatalf("bad: %#v", b.config.FloppyFiles)
+	}
+}
+
 func TestBuilderPrepare_GuestAdditionsPath(t *testing.T) {
 	var b Builder
 	config := testConfig()
 
-	delete(config, "disk_size")
+	delete(config, "guest_additions_path")
 	err := b.Prepare(config)
 	if err != nil {
 		t.Fatalf("bad err: %s", err)
@@ -139,6 +169,81 @@ func TestBuilderPrepare_GuestAdditionsPath(t *testing.T) {
 	}
 }
 
+func TestBuilderPrepare_GuestAdditionsSHA256(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	delete(config, "guest_additions_sha256")
+	err := b.Prepare(config)
+	if err != nil {
+		t.Fatalf("bad err: %s", err)
+	}
+
+	if b.config.GuestAdditionsSHA256 != "" {
+		t.Fatalf("bad: %s", b.config.GuestAdditionsSHA256)
+	}
+
+	config["guest_additions_sha256"] = "FOO"
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	if b.config.GuestAdditionsSHA256 != "foo" {
+		t.Fatalf("bad size: %s", b.config.GuestAdditionsSHA256)
+	}
+}
+
+func TestBuilderPrepare_GuestAdditionsURL(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	config["guest_additions_url"] = ""
+	err := b.Prepare(config)
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+
+	if b.config.GuestAdditionsURL != "" {
+		t.Fatalf("should be empty: %s", b.config.GuestAdditionsURL)
+	}
+
+	config["guest_additions_url"] = "i/am/a/file/that/doesnt/exist"
+	err = b.Prepare(config)
+	if err == nil {
+		t.Error("should have error")
+	}
+
+	config["guest_additions_url"] = "file:i/am/a/file/that/doesnt/exist"
+	err = b.Prepare(config)
+	if err == nil {
+		t.Error("should have error")
+	}
+
+	config["guest_additions_url"] = "http://www.packer.io"
+	err = b.Prepare(config)
+	if err != nil {
+		t.Errorf("should not have error: %s", err)
+	}
+
+	tf, err := ioutil.TempFile("", "packer")
+	if err != nil {
+		t.Fatalf("error tempfile: %s", err)
+	}
+	defer os.Remove(tf.Name())
+
+	config["guest_additions_url"] = tf.Name()
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	if b.config.GuestAdditionsURL != "file://"+tf.Name() {
+		t.Fatalf("guest_additions_url should be modified: %s", b.config.GuestAdditionsURL)
+	}
+}
+
 func TestBuilderPrepare_HTTPPort(t *testing.T) {
 	var b Builder
 	config := testConfig()
@@ -167,26 +272,68 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
 	}
 }
 
-func TestBuilderPrepare_ISOMD5(t *testing.T) {
+func TestBuilderPrepare_InvalidKey(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Add a random key
+	config["i_should_not_be_valid"] = true
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+}
+
+func TestBuilderPrepare_ISOChecksum(t *testing.T) {
 	var b Builder
 	config := testConfig()
 
 	// Test bad
-	config["iso_md5"] = ""
+	config["iso_checksum"] = ""
 	err := b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
 	}
 
 	// Test good
-	config["iso_md5"] = "FOo"
+	config["iso_checksum"] = "FOo"
 	err = b.Prepare(config)
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
 	}
 
-	if b.config.ISOMD5 != "foo" {
-		t.Fatalf("should've lowercased: %s", b.config.ISOMD5)
+	if b.config.ISOChecksum != "foo" {
+		t.Fatalf("should've lowercased: %s", b.config.ISOChecksum)
+	}
+}
+
+func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Test bad
+	config["iso_checksum_type"] = ""
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Test good
+	config["iso_checksum_type"] = "mD5"
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	if b.config.ISOChecksumType != "md5" {
+		t.Fatalf("should've lowercased: %s", b.config.ISOChecksumType)
+	}
+
+	// Test unknown
+	config["iso_checksum_type"] = "fake"
+	err = b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
 	}
 }
 

builder/virtualbox/driver.go

@@ -59,6 +59,12 @@ func (d *VBox42Driver) IsRunning(name string) (bool, error) {
 		if line == `VMState="stopping"` {
 			return true, nil
 		}
+
+		// We consider "paused" to still be running. We wait for it to
+		// be completely stopped or some other state.
+		if line == `VMState="paused"` {
+			return true, nil
+		}
 	}
 
 	return false, nil
@@ -124,11 +130,12 @@ func (d *VBox42Driver) Version() (string, error) {
 		return "", err
 	}
 
-	log.Printf("VBoxManage --version output: %s", stdout.String())
+	versionOutput := strings.TrimSpace(stdout.String())
+	log.Printf("VBoxManage --version output: %s", versionOutput)
 	versionRe := regexp.MustCompile("[^.0-9]")
-	matches := versionRe.Split(stdout.String(), 2)
+	matches := versionRe.Split(versionOutput, 2)
 	if len(matches) == 0 {
-		return "", fmt.Errorf("No version found: %s", stdout.String())
+		return "", fmt.Errorf("No version found: %s", versionOutput)
 	}
 
 	log.Printf("VirtualBox version: %s", matches[0])

builder/virtualbox/ssh.go

@@ -0,0 +1,25 @@
+package virtualbox
+
+import (
+	gossh "code.google.com/p/go.crypto/ssh"
+	"fmt"
+	"github.com/mitchellh/packer/communicator/ssh"
+)
+
+func sshAddress(state map[string]interface{}) (string, error) {
+	sshHostPort := state["sshHostPort"].(uint)
+	return fmt.Sprintf("127.0.0.1:%d", sshHostPort), nil
+}
+
+func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
+	config := state["config"].(*config)
+
+	return &gossh.ClientConfig{
+		User: config.SSHUser,
+		Auth: []gossh.ClientAuth{
+			gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
+			gossh.ClientAuthKeyboardInteractive(
+				ssh.PasswordKeyboardInteractive(config.SSHPassword)),
+		},
+	}, nil
+}

builder/virtualbox/step_attach_floppy.go

@@ -0,0 +1,128 @@
+package virtualbox
+
+import (
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/packer"
+	"io"
+	"io/ioutil"
+	"log"
+	"os"
+	"path/filepath"
+)
+
+// This step attaches the ISO to the virtual machine.
+//
+// Uses:
+//
+// Produces:
+type stepAttachFloppy struct {
+	floppyPath string
+}
+
+func (s *stepAttachFloppy) Run(state map[string]interface{}) multistep.StepAction {
+	// Determine if we even have a floppy disk to attach
+	var floppyPath string
+	if floppyPathRaw, ok := state["floppy_path"]; ok {
+		floppyPath = floppyPathRaw.(string)
+	} else {
+		log.Println("No floppy disk, not attaching.")
+		return multistep.ActionContinue
+	}
+
+	// VirtualBox is really dumb and can't figure out the format of the file
+	// without an extension, so we need to add the "vfd" extension to the
+	// floppy.
+	floppyPath, err := s.copyFloppy(floppyPath)
+	if err != nil {
+		state["error"] = fmt.Errorf("Error preparing floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	driver := state["driver"].(Driver)
+	ui := state["ui"].(packer.Ui)
+	vmName := state["vmName"].(string)
+
+	ui.Say("Attaching floppy disk...")
+
+	// Create the floppy disk controller
+	command := []string{
+		"storagectl", vmName,
+		"--name", "Floppy Controller",
+		"--add", "floppy",
+	}
+	if err := driver.VBoxManage(command...); err != nil {
+		state["error"] = fmt.Errorf("Error creating floppy controller: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// Attach the floppy to the controller
+	command = []string{
+		"storageattach", vmName,
+		"--storagectl", "Floppy Controller",
+		"--port", "0",
+		"--device", "0",
+		"--type", "fdd",
+		"--medium", floppyPath,
+	}
+	if err := driver.VBoxManage(command...); err != nil {
+		state["error"] = fmt.Errorf("Error attaching floppy: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// Track the path so that we can unregister it from VirtualBox later
+	s.floppyPath = floppyPath
+
+	return multistep.ActionContinue
+}
+
+func (s *stepAttachFloppy) Cleanup(state map[string]interface{}) {
+	if s.floppyPath == "" {
+		return
+	}
+
+	// Delete the floppy disk
+	defer os.Remove(s.floppyPath)
+
+	driver := state["driver"].(Driver)
+	vmName := state["vmName"].(string)
+
+	command := []string{
+		"storageattach", vmName,
+		"--storagectl", "Floppy Controller",
+		"--port", "0",
+		"--device", "0",
+		"--medium", "none",
+	}
+
+	if err := driver.VBoxManage(command...); err != nil {
+		log.Printf("Error unregistering floppy: %s", err)
+	}
+}
+
+func (s *stepAttachFloppy) copyFloppy(path string) (string, error) {
+	tempdir, err := ioutil.TempDir("", "packer")
+	if err != nil {
+		return "", err
+	}
+
+	floppyPath := filepath.Join(tempdir, "floppy.vfd")
+	f, err := os.Create(floppyPath)
+	if err != nil {
+		return "", err
+	}
+	defer f.Close()
+
+	sourceF, err := os.Open(path)
+	if err != nil {
+		return "", err
+	}
+	defer sourceF.Close()
+
+	log.Printf("Copying floppy to temp location: %s", floppyPath)
+	if _, err := io.Copy(f, sourceF); err != nil {
+		return "", err
+	}
+
+	return floppyPath, nil
+}

builder/virtualbox/step_create_vm.go

@@ -22,7 +22,10 @@ func (s *stepCreateVM) Run(state map[string]interface{}) multistep.StepAction {
 	name := config.VMName
 
 	commands := make([][]string, 4)
-	commands[0] = []string{"createvm", "--name", name, "--ostype", config.GuestOSType, "--register"}
+	commands[0] = []string{
+		"createvm", "--name", name,
+		"--ostype", config.GuestOSType, "--register",
+	}
 	commands[1] = []string{
 		"modifyvm", name,
 		"--boot1", "disk", "--boot2", "dvd", "--boot3", "none", "--boot4", "none",

builder/virtualbox/step_download_guest_additions.go

@@ -33,7 +33,9 @@ func (s *stepDownloadGuestAdditions) Run(state map[string]interface{}) multistep
 	cache := state["cache"].(packer.Cache)
 	driver := state["driver"].(Driver)
 	ui := state["ui"].(packer.Ui)
+	config := state["config"].(*config)
 
+	// Get VBox version
 	version, err := driver.Version()
 	if err != nil {
 		state["error"] = fmt.Errorf("Error reading version for guest additions download: %s", err)
@@ -45,68 +47,18 @@ func (s *stepDownloadGuestAdditions) Run(state map[string]interface{}) multistep
 		version = newVersion
 	}
 
-	// First things first, we get the list of checksums for the files available
-	// for this version.
-	checksumsUrl := fmt.Sprintf("http://download.virtualbox.org/virtualbox/%s/SHA256SUMS", version)
-	checksumsFile, err := ioutil.TempFile("", "packer")
-	if err != nil {
-		state["error"] = fmt.Errorf(
-			"Failed creating temporary file to store guest addition checksums: %s",
-			err)
-		return multistep.ActionHalt
-	}
-	checksumsFile.Close()
-	defer os.Remove(checksumsFile.Name())
-
-	downloadConfig := &common.DownloadConfig{
-		Url:        checksumsUrl,
-		TargetPath: checksumsFile.Name(),
-		Hash:       nil,
-	}
-
-	log.Printf("Downloading guest addition checksums: %s", checksumsUrl)
-	download := common.NewDownloadClient(downloadConfig)
-	checksumsPath, action := s.progressDownload(download, state)
-	if action != multistep.ActionContinue {
-		return action
-	}
-
 	additionsName := fmt.Sprintf("VBoxGuestAdditions_%s.iso", version)
 
-	// Next, we find the checksum for the file we're looking to download.
-	// It is an error if the checksum cannot be found.
-	checksumsF, err := os.Open(checksumsPath)
-	if err != nil {
-		state["error"] = fmt.Errorf("Error opening guest addition checksums: %s", err)
-		return multistep.ActionHalt
-	}
-	defer checksumsF.Close()
+	// Use provided version or get it from virtualbox.org
+	var checksum string
 
-	// We copy the contents of the file into memory. In general this file
-	// is quite small so that is okay. In the future, we probably want to
-	// use bufio and iterate line by line.
-	var contents bytes.Buffer
-	io.Copy(&contents, checksumsF)
-
-	checksum := ""
-	for _, line := range strings.Split(contents.String(), "\n") {
-		parts := strings.Fields(line)
-		log.Printf("Checksum file parts: %#v", parts)
-		if len(parts) != 2 {
-			// Bogus line
-			continue
+	if config.GuestAdditionsSHA256 != "" {
+		checksum = config.GuestAdditionsSHA256
+	} else {
+		checksum, action = s.downloadAdditionsSHA256(state, version, additionsName)
+		if action != multistep.ActionContinue {
+			return action
 		}
-
-		if strings.HasSuffix(parts[1], additionsName) {
-			checksum = parts[0]
-			log.Printf("Guest additions checksum: %s", checksum)
-			break
-		}
-	}
-
-	if checksum == "" {
-		state["error"] = fmt.Errorf("The checksum for the file '%s' could not be found.", additionsName)
-		return multistep.ActionHalt
 	}
 
 	checksumBytes, err := hex.DecodeString(checksum)
@@ -115,23 +67,29 @@ func (s *stepDownloadGuestAdditions) Run(state map[string]interface{}) multistep
 		return multistep.ActionHalt
 	}
 
-	url := fmt.Sprintf(
-		"http://download.virtualbox.org/virtualbox/%s/%s",
-		version, additionsName)
+	// Use the provided source (URL or file path) or generate it
+	url := config.GuestAdditionsURL
+	if url == "" {
+		url = fmt.Sprintf(
+			"http://download.virtualbox.org/virtualbox/%s/%s",
+			version,
+			additionsName)
+	}
+
 	log.Printf("Guest additions URL: %s", url)
 
 	log.Printf("Acquiring lock to download the guest additions ISO.")
 	cachePath := cache.Lock(url)
 	defer cache.Unlock(url)
 
-	downloadConfig = &common.DownloadConfig{
+	downloadConfig := &common.DownloadConfig{
 		Url:        url,
 		TargetPath: cachePath,
 		Hash:       sha256.New(),
 		Checksum:   checksumBytes,
 	}
 
-	download = common.NewDownloadClient(downloadConfig)
+	download := common.NewDownloadClient(downloadConfig)
 	ui.Say("Downloading VirtualBox guest additions. Progress will be shown periodically.")
 	state["guest_additions_path"], action = s.progressDownload(download, state)
 	return action
@@ -179,3 +137,72 @@ DownloadWaitLoop:
 
 	return result, multistep.ActionContinue
 }
+
+func (s *stepDownloadGuestAdditions) downloadAdditionsSHA256(state map[string]interface{}, additionsVersion string, additionsName string) (string, multistep.StepAction) {
+	// First things first, we get the list of checksums for the files available
+	// for this version.
+	checksumsUrl := fmt.Sprintf("http://download.virtualbox.org/virtualbox/%s/SHA256SUMS", additionsVersion)
+
+	checksumsFile, err := ioutil.TempFile("", "packer")
+	if err != nil {
+		state["error"] = fmt.Errorf(
+			"Failed creating temporary file to store guest addition checksums: %s",
+			err)
+		return "", multistep.ActionHalt
+	}
+	defer os.Remove(checksumsFile.Name())
+
+	checksumsFile.Close()
+
+	downloadConfig := &common.DownloadConfig{
+		Url:        checksumsUrl,
+		TargetPath: checksumsFile.Name(),
+		Hash:       nil,
+	}
+
+	log.Printf("Downloading guest addition checksums: %s", checksumsUrl)
+	download := common.NewDownloadClient(downloadConfig)
+	checksumsPath, action := s.progressDownload(download, state)
+	if action != multistep.ActionContinue {
+		return "", action
+	}
+
+	// Next, we find the checksum for the file we're looking to download.
+	// It is an error if the checksum cannot be found.
+	checksumsF, err := os.Open(checksumsPath)
+	if err != nil {
+		state["error"] = fmt.Errorf("Error opening guest addition checksums: %s", err)
+		return "", multistep.ActionHalt
+	}
+	defer checksumsF.Close()
+
+	// We copy the contents of the file into memory. In general this file
+	// is quite small so that is okay. In the future, we probably want to
+	// use bufio and iterate line by line.
+	var contents bytes.Buffer
+	io.Copy(&contents, checksumsF)
+
+	checksum := ""
+	for _, line := range strings.Split(contents.String(), "\n") {
+		parts := strings.Fields(line)
+		log.Printf("Checksum file parts: %#v", parts)
+		if len(parts) != 2 {
+			// Bogus line
+			continue
+		}
+
+		if strings.HasSuffix(parts[1], additionsName) {
+			checksum = parts[0]
+			log.Printf("Guest additions checksum: %s", checksum)
+			break
+		}
+	}
+
+	if checksum == "" {
+		state["error"] = fmt.Errorf("The checksum for the file '%s' could not be found.", additionsName)
+		return "", multistep.ActionHalt
+	}
+
+	return checksum, multistep.ActionContinue
+
+}

builder/virtualbox/step_download_iso.go

@@ -1,7 +1,6 @@
 package virtualbox
 
 import (
-	"crypto/md5"
 	"encoding/hex"
 	"fmt"
 	"github.com/mitchellh/multistep"
@@ -33,7 +32,7 @@ func (s *stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
 	config := state["config"].(*config)
 	ui := state["ui"].(packer.Ui)
 
-	checksum, err := hex.DecodeString(config.ISOMD5)
+	checksum, err := hex.DecodeString(config.ISOChecksum)
 	if err != nil {
 		state["error"] = fmt.Errorf("Error parsing checksum: %s", err)
 		return multistep.ActionHalt
@@ -47,7 +46,7 @@ func (s *stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
 		Url:        config.ISOUrl,
 		TargetPath: cachePath,
 		CopyFile:   false,
-		Hash:       md5.New(),
+		Hash:       common.HashForType(config.ISOChecksumType),
 		Checksum:   checksum,
 	}
 

builder/virtualbox/step_export.go

@@ -7,8 +7,7 @@ import (
 	"path/filepath"
 )
 
-// This step creates the virtual disk that will be used as the
-// hard drive for the virtual machine.
+// This step cleans up forwarded ports and exports the VM to an OVF.
 //
 // Uses:
 //
@@ -22,9 +21,38 @@ func (s *stepExport) Run(state map[string]interface{}) multistep.StepAction {
 	ui := state["ui"].(packer.Ui)
 	vmName := state["vmName"].(string)
 
+	// Clear out the Packer-created forwarding rule
+	ui.Say("Preparing to export machine...")
+	ui.Message(fmt.Sprintf("Deleting forwarded port mapping for SSH (host port %d)", state["sshHostPort"]))
+	command := []string{"modifyvm", vmName, "--natpf1", "delete", "packerssh"}
+	if err := driver.VBoxManage(command...); err != nil {
+		err := fmt.Errorf("Error deleting port forwarding rule: %s", err)
+		state["error"] = err
+		ui.Error(err.Error())
+		return multistep.ActionHalt
+	}
+
+	// Remove the attached floppy disk, if it exists
+	if _, ok := state["floppy_path"]; ok {
+		ui.Message("Removing floppy drive...")
+		command := []string{
+			"storageattach", vmName,
+			"--storagectl", "Floppy Controller",
+			"--port", "0",
+			"--device", "0",
+			"--medium", "none",
+		}
+		if err := driver.VBoxManage(command...); err != nil {
+			state["error"] = fmt.Errorf("Error removing floppy: %s", err)
+			return multistep.ActionHalt
+		}
+
+	}
+
+	// Export the VM to an OVF
 	outputPath := filepath.Join(config.OutputDir, "packer.ovf")
 
-	command := []string{
+	command = []string{
 		"export",
 		vmName,
 		"--output",

builder/virtualbox/step_forward_ssh.go

@@ -36,7 +36,7 @@ func (s *stepForwardSSH) Run(state map[string]interface{}) multistep.StepAction
 		}
 	}
 
-	// Attach the disk to the controller
+	// Create a forwarded port mapping to the VM
 	ui.Say(fmt.Sprintf("Creating forwarded port mapping for SSH (host port %d)", sshHostPort))
 	command := []string{
 		"modifyvm", vmName,

builder/virtualbox/step_prepare_output_dir.go

@@ -10,6 +10,12 @@ type stepPrepareOutputDir struct{}
 
 func (stepPrepareOutputDir) Run(state map[string]interface{}) multistep.StepAction {
 	config := state["config"].(*config)
+	ui := state["ui"].(packer.Ui)
+
+	if _, err := os.Stat(config.OutputDir); err == nil && config.PackerForce {
+		ui.Say("Deleting previous output directory...")
+		os.RemoveAll(config.OutputDir)
+	}
 
 	if err := os.MkdirAll(config.OutputDir, 0755); err != nil {
 		state["error"] = err

builder/virtualbox/step_provision.go

@@ -1,25 +0,0 @@
-package virtualbox
-
-import (
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/packer"
-	"log"
-)
-
-type stepProvision struct{}
-
-func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
-	comm := state["communicator"].(packer.Communicator)
-	hook := state["hook"].(packer.Hook)
-	ui := state["ui"].(packer.Ui)
-
-	log.Println("Running the provision hook")
-	if err := hook.Run(packer.HookProvision, ui, comm, nil); err != nil {
-		state["error"] = err
-		return multistep.ActionHalt
-	}
-
-	return multistep.ActionContinue
-}
-
-func (*stepProvision) Cleanup(map[string]interface{}) {}

builder/virtualbox/step_run.go

@@ -23,7 +23,14 @@ func (s *stepRun) Run(state map[string]interface{}) multistep.StepAction {
 	vmName := state["vmName"].(string)
 
 	ui.Say("Starting the virtual machine...")
-	command := []string{"startvm", vmName, "--type", "gui"}
+	guiArgument := "gui"
+	if config.Headless == true {
+		ui.Message("WARNING: The VM will be started in headless mode, as configured.\n" +
+			"In headless mode, errors during the boot sequence or OS setup\n" +
+			"won't be easily visible. Use at your own discretion.")
+		guiArgument = "headless"
+	}
+	command := []string{"startvm", vmName, "--type", guiArgument}
 	if err := driver.VBoxManage(command...); err != nil {
 		err := fmt.Errorf("Error starting VM: %s", err)
 		state["error"] = err
@@ -33,9 +40,9 @@ func (s *stepRun) Run(state map[string]interface{}) multistep.StepAction {
 
 	s.vmName = vmName
 
-	if int64(config.BootWait) > 0 {
-		ui.Say(fmt.Sprintf("Waiting %s for boot...", config.BootWait))
-		time.Sleep(config.BootWait)
+	if int64(config.bootWait) > 0 {
+		ui.Say(fmt.Sprintf("Waiting %s for boot...", config.bootWait))
+		time.Sleep(config.bootWait)
 	}
 
 	return multistep.ActionContinue

builder/virtualbox/step_shutdown.go

@@ -45,8 +45,8 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
 		cmd.Wait()
 
 		// Wait for the machine to actually shut down
-		log.Printf("Waiting max %s for shutdown to complete", config.ShutdownTimeout)
-		shutdownTimer := time.After(config.ShutdownTimeout)
+		log.Printf("Waiting max %s for shutdown to complete", config.shutdownTimeout)
+		shutdownTimer := time.After(config.shutdownTimeout)
 		for {
 			running, _ := driver.IsRunning(vmName)
 			if !running {

builder/virtualbox/step_type_boot_command.go

@@ -59,6 +59,16 @@ func (s *stepTypeBootCommand) Run(state map[string]interface{}) multistep.StepAc
 				continue
 			}
 
+			if code == "wait5" {
+				time.Sleep(5 * time.Second)
+				continue
+			}
+
+			if code == "wait10" {
+				time.Sleep(10 * time.Second)
+				continue
+			}
+
 			// Since typing is sometimes so slow, we check for an interrupt
 			// in between each character.
 			if _, ok := state[multistep.StateCancelled]; ok {
@@ -116,11 +126,23 @@ func scancodes(message string) []string {
 		var scancode []string
 
 		if strings.HasPrefix(message, "<wait>") {
-			log.Printf("Special code <wait> found, will sleep at this point.")
+			log.Printf("Special code <wait> found, will sleep 1 second at this point.")
 			scancode = []string{"wait"}
 			message = message[len("<wait>"):]
 		}
 
+		if strings.HasPrefix(message, "<wait5>") {
+			log.Printf("Special code <wait5> found, will sleep 5 seconds at this point.")
+			scancode = []string{"wait5"}
+			message = message[len("<wait5>"):]
+		}
+
+		if strings.HasPrefix(message, "<wait10>") {
+			log.Printf("Special code <wait10> found, will sleep 10 seconds at this point.")
+			scancode = []string{"wait10"}
+			message = message[len("<wait10>"):]
+		}
+
 		if scancode == nil {
 			for specialCode, specialValue := range special {
 				if strings.HasPrefix(message, specialCode) {

builder/virtualbox/step_wait_for_ssh.go

@@ -1,147 +0,0 @@
-package virtualbox
-
-import (
-	gossh "code.google.com/p/go.crypto/ssh"
-	"errors"
-	"fmt"
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/communicator/ssh"
-	"github.com/mitchellh/packer/packer"
-	"log"
-	"net"
-	"time"
-)
-
-// This step waits for SSH to become available and establishes an SSH
-// connection.
-//
-// Uses:
-//   config *config
-//   sshHostPort uint
-//   ui     packer.Ui
-//
-// Produces:
-//   communicator packer.Communicator
-type stepWaitForSSH struct {
-	cancel bool
-	conn   net.Conn
-}
-
-func (s *stepWaitForSSH) Run(state map[string]interface{}) multistep.StepAction {
-	config := state["config"].(*config)
-	ui := state["ui"].(packer.Ui)
-
-	var comm packer.Communicator
-	var err error
-
-	waitDone := make(chan bool, 1)
-	go func() {
-		comm, err = s.waitForSSH(state)
-		waitDone <- true
-	}()
-
-	log.Printf("Waiting for SSH, up to timeout: %s", config.SSHWaitTimeout.String())
-
-	timeout := time.After(config.SSHWaitTimeout)
-WaitLoop:
-	for {
-		// Wait for either SSH to become available, a timeout to occur,
-		// or an interrupt to come through.
-		select {
-		case <-waitDone:
-			if err != nil {
-				ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
-				return multistep.ActionHalt
-			}
-
-			state["communicator"] = comm
-			break WaitLoop
-		case <-timeout:
-			ui.Error("Timeout waiting for SSH.")
-			s.cancel = true
-			return multistep.ActionHalt
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				log.Println("Interrupt detected, quitting waiting for SSH.")
-				return multistep.ActionHalt
-			}
-		}
-	}
-
-	return multistep.ActionContinue
-}
-
-func (s *stepWaitForSSH) Cleanup(map[string]interface{}) {
-	if s.conn != nil {
-		s.conn.Close()
-		s.conn = nil
-	}
-}
-
-// This blocks until SSH becomes available, and sends the communicator
-// on the given channel.
-func (s *stepWaitForSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
-	config := state["config"].(*config)
-	ui := state["ui"].(packer.Ui)
-	sshHostPort := state["sshHostPort"].(uint)
-
-	ui.Say("Waiting for SSH to become available...")
-	var comm packer.Communicator
-	var nc net.Conn
-	for {
-		if nc != nil {
-			nc.Close()
-		}
-
-		time.Sleep(5 * time.Second)
-
-		if s.cancel {
-			log.Println("SSH wait cancelled. Exiting loop.")
-			return nil, errors.New("SSH wait cancelled")
-		}
-
-		// Attempt to connect to SSH port
-		nc, err := net.Dial("tcp", fmt.Sprintf("127.0.0.1:%d", sshHostPort))
-		if err != nil {
-			log.Printf("TCP connection to SSH ip/port failed: %s", err)
-			continue
-		}
-
-		// Then we attempt to connect via SSH
-		sshConfig := &gossh.ClientConfig{
-			User: config.SSHUser,
-			Auth: []gossh.ClientAuth{
-				gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
-			},
-		}
-
-		sshConnectSuccess := make(chan bool, 1)
-		go func() {
-			comm, err = ssh.New(nc, sshConfig)
-			if err != nil {
-				log.Printf("SSH connection fail: %s", err)
-				sshConnectSuccess <- false
-				return
-			}
-
-			sshConnectSuccess <- true
-		}()
-
-		select {
-		case success := <-sshConnectSuccess:
-			if !success {
-				continue
-			}
-		case <-time.After(5 * time.Second):
-			log.Printf("SSH handshake timeout. Trying again.")
-			continue
-		}
-
-		ui.Say("Connected via SSH!")
-		break
-	}
-
-	// Store the connection so we can close it later
-	s.conn = nc
-	return comm, nil
-}

builder/vmware/builder.go

@@ -12,7 +12,9 @@ import (
 	"net/url"
 	"os"
 	"path/filepath"
+	"sort"
 	"strings"
+	"text/template"
 	"time"
 )
 
@@ -25,43 +27,77 @@ type Builder struct {
 }
 
 type config struct {
-	DiskName        string            `mapstructure:"vmdk_name"`
-	DiskSize        uint              `mapstructure:"disk_size"`
-	GuestOSType     string            `mapstructure:"guest_os_type"`
-	ISOMD5          string            `mapstructure:"iso_md5"`
-	ISOUrl          string            `mapstructure:"iso_url"`
-	VMName          string            `mapstructure:"vm_name"`
-	OutputDir       string            `mapstructure:"output_directory"`
-	HTTPDir         string            `mapstructure:"http_directory"`
-	HTTPPortMin     uint              `mapstructure:"http_port_min"`
-	HTTPPortMax     uint              `mapstructure:"http_port_max"`
-	BootCommand     []string          `mapstructure:"boot_command"`
-	BootWait        time.Duration     ``
-	ShutdownCommand string            `mapstructure:"shutdown_command"`
-	ShutdownTimeout time.Duration     ``
-	SSHUser         string            `mapstructure:"ssh_username"`
-	SSHPassword     string            `mapstructure:"ssh_password"`
-	SSHPort         uint              `mapstructure:"ssh_port"`
-	SSHWaitTimeout  time.Duration     ``
-	VMXData         map[string]string `mapstructure:"vmx_data"`
-	VNCPortMin      uint              `mapstructure:"vnc_port_min"`
-	VNCPortMax      uint              `mapstructure:"vnc_port_max"`
+	DiskName          string            `mapstructure:"vmdk_name"`
+	DiskSize          uint              `mapstructure:"disk_size"`
+	FloppyFiles       []string          `mapstructure:"floppy_files"`
+	GuestOSType       string            `mapstructure:"guest_os_type"`
+	ISOChecksum       string            `mapstructure:"iso_checksum"`
+	ISOChecksumType   string            `mapstructure:"iso_checksum_type"`
+	ISOUrl            string            `mapstructure:"iso_url"`
+	VMName            string            `mapstructure:"vm_name"`
+	OutputDir         string            `mapstructure:"output_directory"`
+	Headless          bool              `mapstructure:"headless"`
+	HTTPDir           string            `mapstructure:"http_directory"`
+	HTTPPortMin       uint              `mapstructure:"http_port_min"`
+	HTTPPortMax       uint              `mapstructure:"http_port_max"`
+	BootCommand       []string          `mapstructure:"boot_command"`
+	SkipCompaction    bool              `mapstructure:"skip_compaction"`
+	ShutdownCommand   string            `mapstructure:"shutdown_command"`
+	SSHUser           string            `mapstructure:"ssh_username"`
+	SSHPassword       string            `mapstructure:"ssh_password"`
+	SSHPort           uint              `mapstructure:"ssh_port"`
+	ToolsUploadFlavor string            `mapstructure:"tools_upload_flavor"`
+	ToolsUploadPath   string            `mapstructure:"tools_upload_path"`
+	VMXData           map[string]string `mapstructure:"vmx_data"`
+	VNCPortMin        uint              `mapstructure:"vnc_port_min"`
+	VNCPortMax        uint              `mapstructure:"vnc_port_max"`
 
-	PackerDebug bool `mapstructure:"packer_debug"`
+	PackerBuildName string `mapstructure:"packer_build_name"`
+	PackerDebug     bool   `mapstructure:"packer_debug"`
+	PackerForce     bool   `mapstructure:"packer_force"`
 
 	RawBootWait        string `mapstructure:"boot_wait"`
 	RawShutdownTimeout string `mapstructure:"shutdown_timeout"`
 	RawSSHWaitTimeout  string `mapstructure:"ssh_wait_timeout"`
+
+	bootWait        time.Duration ``
+	shutdownTimeout time.Duration ``
+	sshWaitTimeout  time.Duration ``
 }
 
 func (b *Builder) Prepare(raws ...interface{}) error {
+	var md mapstructure.Metadata
+	decoderConfig := &mapstructure.DecoderConfig{
+		Metadata: &md,
+		Result:   &b.config,
+	}
+
+	decoder, err := mapstructure.NewDecoder(decoderConfig)
+	if err != nil {
+		return err
+	}
+
 	for _, raw := range raws {
-		err := mapstructure.Decode(raw, &b.config)
+		err := decoder.Decode(raw)
 		if err != nil {
 			return err
 		}
 	}
 
+	// Accumulate any errors
+	errs := make([]error, 0)
+
+	// Unused keys are errors
+	if len(md.Unused) > 0 {
+		sort.Strings(md.Unused)
+		for _, unused := range md.Unused {
+			if unused != "type" && !strings.HasPrefix(unused, "packer_") {
+				errs = append(
+					errs, fmt.Errorf("Unknown configuration key: %s", unused))
+			}
+		}
+	}
+
 	if b.config.DiskName == "" {
 		b.config.DiskName = "disk"
 	}
@@ -70,12 +106,16 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.DiskSize = 40000
 	}
 
+	if b.config.FloppyFiles == nil {
+		b.config.FloppyFiles = make([]string, 0)
+	}
+
 	if b.config.GuestOSType == "" {
 		b.config.GuestOSType = "other"
 	}
 
 	if b.config.VMName == "" {
-		b.config.VMName = "packer"
+		b.config.VMName = fmt.Sprintf("packer-%s", b.config.PackerBuildName)
 	}
 
 	if b.config.HTTPPortMin == 0 {
@@ -99,25 +139,36 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 	}
 
 	if b.config.OutputDir == "" {
-		b.config.OutputDir = "vmware"
+		b.config.OutputDir = fmt.Sprintf("output-%s", b.config.PackerBuildName)
 	}
 
 	if b.config.SSHPort == 0 {
 		b.config.SSHPort = 22
 	}
 
-	// Accumulate any errors
-	var err error
-	errs := make([]error, 0)
+	if b.config.ToolsUploadPath == "" {
+		b.config.ToolsUploadPath = "{{ .Flavor }}.iso"
+	}
 
 	if b.config.HTTPPortMin > b.config.HTTPPortMax {
 		errs = append(errs, errors.New("http_port_min must be less than http_port_max"))
 	}
 
-	if b.config.ISOMD5 == "" {
-		errs = append(errs, errors.New("Due to large file sizes, an iso_md5 is required"))
+	if b.config.ISOChecksum == "" {
+		errs = append(errs, errors.New("Due to large file sizes, an iso_checksum is required"))
 	} else {
-		b.config.ISOMD5 = strings.ToLower(b.config.ISOMD5)
+		b.config.ISOChecksum = strings.ToLower(b.config.ISOChecksum)
+	}
+
+	if b.config.ISOChecksumType == "" {
+		errs = append(errs, errors.New("The iso_checksum_type must be specified."))
+	} else {
+		b.config.ISOChecksumType = strings.ToLower(b.config.ISOChecksumType)
+		if h := common.HashForType(b.config.ISOChecksumType); h == nil {
+			errs = append(
+				errs,
+				fmt.Errorf("Unsupported checksum type: %s", b.config.ISOChecksumType))
+		}
 	}
 
 	if b.config.ISOUrl == "" {
@@ -159,8 +210,12 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		}
 	}
 
-	if _, err := os.Stat(b.config.OutputDir); err == nil {
-		errs = append(errs, errors.New("Output directory already exists. It must not exist."))
+	if !b.config.PackerForce {
+		if _, err := os.Stat(b.config.OutputDir); err == nil {
+			errs = append(
+				errs,
+				errors.New("Output directory already exists. It must not exist."))
+		}
 	}
 
 	if b.config.SSHUser == "" {
@@ -168,7 +223,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 	}
 
 	if b.config.RawBootWait != "" {
-		b.config.BootWait, err = time.ParseDuration(b.config.RawBootWait)
+		b.config.bootWait, err = time.ParseDuration(b.config.RawBootWait)
 		if err != nil {
 			errs = append(errs, fmt.Errorf("Failed parsing boot_wait: %s", err))
 		}
@@ -178,7 +233,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.RawShutdownTimeout = "5m"
 	}
 
-	b.config.ShutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
+	b.config.shutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing shutdown_timeout: %s", err))
 	}
@@ -187,11 +242,15 @@ func (b *Builder) Prepare(raws ...interface{}) error {
 		b.config.RawSSHWaitTimeout = "20m"
 	}
 
-	b.config.SSHWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
+	b.config.sshWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
 	if err != nil {
 		errs = append(errs, fmt.Errorf("Failed parsing ssh_wait_timeout: %s", err))
 	}
 
+	if _, err := template.New("path").Parse(b.config.ToolsUploadPath); err != nil {
+		errs = append(errs, fmt.Errorf("tools_upload_path invalid: %s", err))
+	}
+
 	if b.config.VNCPortMin > b.config.VNCPortMax {
 		errs = append(errs, fmt.Errorf("vnc_port_min must be less than vnc_port_max"))
 	}
@@ -213,18 +272,29 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 	rand.Seed(time.Now().UTC().UnixNano())
 
 	steps := []multistep.Step{
+		&stepPrepareTools{},
 		&stepDownloadISO{},
 		&stepPrepareOutputDir{},
+		&common.StepCreateFloppy{
+			Files: b.config.FloppyFiles,
+		},
 		&stepCreateDisk{},
 		&stepCreateVMX{},
 		&stepHTTPServer{},
 		&stepConfigureVNC{},
 		&stepRun{},
 		&stepTypeBootCommand{},
-		&stepWaitForSSH{},
-		&stepProvision{},
+		&common.StepConnectSSH{
+			SSHAddress:     sshAddress,
+			SSHConfig:      sshConfig,
+			SSHWaitTimeout: b.config.sshWaitTimeout,
+		},
+		&stepUploadTools{},
+		&common.StepProvision{},
 		&stepShutdown{},
 		&stepCleanFiles{},
+		&stepCleanVMX{},
+		&stepCompactDisk{},
 	}
 
 	// Setup the state bag

builder/vmware/builder_test.go

@@ -4,15 +4,19 @@ import (
 	"github.com/mitchellh/packer/packer"
 	"io/ioutil"
 	"os"
+	"reflect"
 	"testing"
 	"time"
 )
 
 func testConfig() map[string]interface{} {
 	return map[string]interface{}{
-		"iso_md5":      "foo",
-		"iso_url":      "http://www.packer.io",
-		"ssh_username": "foo",
+		"iso_checksum":      "foo",
+		"iso_checksum_type": "md5",
+		"iso_url":           "http://www.packer.io",
+		"ssh_username":      "foo",
+
+		packer.BuildNameConfigKey: "foo",
 	}
 }
 
@@ -54,6 +58,58 @@ func TestBuilderPrepare_BootWait(t *testing.T) {
 	}
 }
 
+func TestBuilderPrepare_ISOChecksum(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Test bad
+	config["iso_checksum"] = ""
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Test good
+	config["iso_checksum"] = "FOo"
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	if b.config.ISOChecksum != "foo" {
+		t.Fatalf("should've lowercased: %s", b.config.ISOChecksum)
+	}
+}
+
+func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Test bad
+	config["iso_checksum_type"] = ""
+	err := b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Test good
+	config["iso_checksum_type"] = "mD5"
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	if b.config.ISOChecksumType != "md5" {
+		t.Fatalf("should've lowercased: %s", b.config.ISOChecksumType)
+	}
+
+	// Test unknown
+	config["iso_checksum_type"] = "fake"
+	err = b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+}
 func TestBuilderPrepare_Defaults(t *testing.T) {
 	var b Builder
 	config := testConfig()
@@ -66,15 +122,15 @@ func TestBuilderPrepare_Defaults(t *testing.T) {
 		t.Errorf("bad disk name: %s", b.config.DiskName)
 	}
 
-	if b.config.OutputDir != "vmware" {
+	if b.config.OutputDir != "output-foo" {
 		t.Errorf("bad output dir: %s", b.config.OutputDir)
 	}
 
-	if b.config.SSHWaitTimeout != (20 * time.Minute) {
-		t.Errorf("bad wait timeout: %s", b.config.SSHWaitTimeout)
+	if b.config.sshWaitTimeout != (20 * time.Minute) {
+		t.Errorf("bad wait timeout: %s", b.config.sshWaitTimeout)
 	}
 
-	if b.config.VMName != "packer" {
+	if b.config.VMName != "packer-foo" {
 		t.Errorf("bad vm name: %s", b.config.VMName)
 	}
 }
@@ -105,6 +161,33 @@ func TestBuilderPrepare_DiskSize(t *testing.T) {
 	}
 }
 
+func TestBuilderPrepare_FloppyFiles(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	delete(config, "floppy_files")
+	err := b.Prepare(config)
+	if err != nil {
+		t.Fatalf("bad err: %s", err)
+	}
+
+	if len(b.config.FloppyFiles) != 0 {
+		t.Fatalf("bad: %#v", b.config.FloppyFiles)
+	}
+
+	config["floppy_files"] = []string{"foo", "bar"}
+	b = Builder{}
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+
+	expected := []string{"foo", "bar"}
+	if !reflect.DeepEqual(b.config.FloppyFiles, expected) {
+		t.Fatalf("bad: %#v", b.config.FloppyFiles)
+	}
+}
+
 func TestBuilderPrepare_HTTPPort(t *testing.T) {
 	var b Builder
 	config := testConfig()
@@ -133,27 +216,16 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
 	}
 }
 
-func TestBuilderPrepare_ISOMD5(t *testing.T) {
+func TestBuilderPrepare_InvalidKey(t *testing.T) {
 	var b Builder
 	config := testConfig()
 
-	// Test bad
-	config["iso_md5"] = ""
+	// Add a random key
+	config["i_should_not_be_valid"] = true
 	err := b.Prepare(config)
 	if err == nil {
 		t.Fatal("should have error")
 	}
-
-	// Test good
-	config["iso_md5"] = "FOo"
-	err = b.Prepare(config)
-	if err != nil {
-		t.Fatalf("should not have error: %s", err)
-	}
-
-	if b.config.ISOMD5 != "foo" {
-		t.Fatalf("should've lowercased: %s", b.config.ISOMD5)
-	}
 }
 
 func TestBuilderPrepare_ISOUrl(t *testing.T) {
@@ -309,6 +381,36 @@ func TestBuilderPrepare_SSHWaitTimeout(t *testing.T) {
 	}
 }
 
+func TestBuilderPrepare_ToolsUploadPath(t *testing.T) {
+	var b Builder
+	config := testConfig()
+
+	// Test a default
+	delete(config, "tools_upload_path")
+	err := b.Prepare(config)
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+
+	if b.config.ToolsUploadPath == "" {
+		t.Fatalf("bad value: %s", b.config.ToolsUploadPath)
+	}
+
+	// Test with a bad value
+	config["tools_upload_path"] = "{{{nope}"
+	err = b.Prepare(config)
+	if err == nil {
+		t.Fatal("should have error")
+	}
+
+	// Test with a good one
+	config["tools_upload_path"] = "hey"
+	err = b.Prepare(config)
+	if err != nil {
+		t.Fatalf("should not have error: %s", err)
+	}
+}
+
 func TestBuilderPrepare_VNCPort(t *testing.T) {
 	var b Builder
 	config := testConfig()

builder/vmware/driver.go

@@ -12,6 +12,9 @@ import (
 
 // A driver is able to talk to VMware, control virtual machines, etc.
 type Driver interface {
+	// CompactDisk compacts a virtual disk.
+	CompactDisk(string) error
+
 	// CreateDisk creates a virtual disk with the given size.
 	CreateDisk(string, string) error
 
@@ -19,11 +22,14 @@ type Driver interface {
 	IsRunning(string) (bool, error)
 
 	// Start starts a VM specified by the path to the VMX given.
-	Start(string) error
+	Start(string, bool) error
 
 	// Stop stops a VM specified by the path to the VMX given.
 	Stop(string) error
 
+	// Get the path to the VMware ISO for the given flavor.
+	ToolsIsoPath(string) string
+
 	// Verify checks to make sure that this driver should function
 	// properly. This should check that all the files it will use
 	// appear to exist and so on. If everything is okay, this doesn't
@@ -37,6 +43,20 @@ type Fusion5Driver struct {
 	AppPath string
 }
 
+func (d *Fusion5Driver) CompactDisk(diskPath string) error {
+	defragCmd := exec.Command(d.vdiskManagerPath(), "-d", diskPath)
+	if _, _, err := d.runAndLog(defragCmd); err != nil {
+		return err
+	}
+
+	shrinkCmd := exec.Command(d.vdiskManagerPath(), "-k", diskPath)
+	if _, _, err := d.runAndLog(shrinkCmd); err != nil {
+		return err
+	}
+
+	return nil
+}
+
 func (d *Fusion5Driver) CreateDisk(output string, size string) error {
 	cmd := exec.Command(d.vdiskManagerPath(), "-c", "-s", size, "-a", "lsilogic", "-t", "1", output)
 	if _, _, err := d.runAndLog(cmd); err != nil {
@@ -67,8 +87,13 @@ func (d *Fusion5Driver) IsRunning(vmxPath string) (bool, error) {
 	return false, nil
 }
 
-func (d *Fusion5Driver) Start(vmxPath string) error {
-	cmd := exec.Command(d.vmrunPath(), "-T", "fusion", "start", vmxPath, "gui")
+func (d *Fusion5Driver) Start(vmxPath string, headless bool) error {
+	guiArgument := "gui"
+	if headless == true {
+		guiArgument = "nogui"
+	}
+
+	cmd := exec.Command(d.vmrunPath(), "-T", "fusion", "start", vmxPath, guiArgument)
 	if _, _, err := d.runAndLog(cmd); err != nil {
 		return err
 	}
@@ -121,6 +146,10 @@ func (d *Fusion5Driver) vmrunPath() string {
 	return filepath.Join(d.AppPath, "Contents", "Library", "vmrun")
 }
 
+func (d *Fusion5Driver) ToolsIsoPath(k string) string {
+	return filepath.Join(d.AppPath, "Contents", "Library", "isoimages", k+".iso")
+}
+
 func (d *Fusion5Driver) runAndLog(cmd *exec.Cmd) (string, string, error) {
 	var stdout, stderr bytes.Buffer
 

builder/vmware/guest_ip.go

@@ -1,6 +1,7 @@
 package vmware
 
 import (
+	"errors"
 	"fmt"
 	"io/ioutil"
 	"os"
@@ -68,5 +69,9 @@ func (f *DHCPLeaseGuestLookup) GuestIP() (string, error) {
 		}
 	}
 
+	if curIp == "" {
+		return "", errors.New("IP not found for MAC in DHCP leases")
+	}
+
 	return curIp, nil
 }

builder/vmware/ssh.go

@@ -0,0 +1,70 @@
+package vmware
+
+import (
+	gossh "code.google.com/p/go.crypto/ssh"
+	"errors"
+	"fmt"
+	"github.com/mitchellh/packer/communicator/ssh"
+	"io/ioutil"
+	"log"
+	"os"
+)
+
+func sshAddress(state map[string]interface{}) (string, error) {
+	config := state["config"].(*config)
+	vmxPath := state["vmx_path"].(string)
+
+	log.Println("Lookup up IP information...")
+	f, err := os.Open(vmxPath)
+	if err != nil {
+		return "", err
+	}
+	defer f.Close()
+
+	vmxBytes, err := ioutil.ReadAll(f)
+	if err != nil {
+		return "", err
+	}
+
+	vmxData := ParseVMX(string(vmxBytes))
+
+	var ok bool
+	macAddress := ""
+	if macAddress, ok = vmxData["ethernet0.address"]; !ok || macAddress == "" {
+		if macAddress, ok = vmxData["ethernet0.generatedAddress"]; !ok || macAddress == "" {
+			return "", errors.New("couldn't find MAC address in VMX")
+		}
+	}
+
+	ipLookup := &DHCPLeaseGuestLookup{
+		Device:     "vmnet8",
+		MACAddress: macAddress,
+	}
+
+	ipAddress, err := ipLookup.GuestIP()
+	if err != nil {
+		log.Printf("IP lookup failed: %s", err)
+		return "", fmt.Errorf("IP lookup failed: %s", err)
+	}
+
+	if ipAddress == "" {
+		log.Println("IP is blank, no IP yet.")
+		return "", errors.New("IP is blank")
+	}
+
+	log.Printf("Detected IP: %s", ipAddress)
+	return fmt.Sprintf("%s:%d", ipAddress, config.SSHPort), nil
+}
+
+func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
+	config := state["config"].(*config)
+
+	return &gossh.ClientConfig{
+		User: config.SSHUser,
+		Auth: []gossh.ClientAuth{
+			gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
+			gossh.ClientAuthKeyboardInteractive(
+				ssh.PasswordKeyboardInteractive(config.SSHPassword)),
+		},
+	}, nil
+}

builder/vmware/step_clean_vmx.go

@@ -0,0 +1,72 @@
+package vmware
+
+import (
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/packer"
+	"io/ioutil"
+	"log"
+	"os"
+	"strings"
+)
+
+// This step cleans up the VMX by removing or changing this prior to
+// being ready for use.
+//
+// Uses:
+//   ui     packer.Ui
+//   vmx_path string
+//
+// Produces:
+//   <nothing>
+type stepCleanVMX struct{}
+
+func (s stepCleanVMX) Run(state map[string]interface{}) multistep.StepAction {
+	if _, ok := state["floppy_path"]; !ok {
+		return multistep.ActionContinue
+	}
+
+	ui := state["ui"].(packer.Ui)
+	vmxPath := state["vmx_path"].(string)
+
+	vmxData, err := s.readVMX(vmxPath)
+	if err != nil {
+		state["error"] = fmt.Errorf("Error reading VMX: %s", err)
+		return multistep.ActionHalt
+	}
+
+	// Delete the floppy0 entries so the floppy is no longer mounted
+	ui.Say("Unmounting floppy from VMX...")
+	for k, _ := range vmxData {
+		if strings.HasPrefix(k, "floppy0.") {
+			log.Printf("Deleting key: %s", k)
+			delete(vmxData, k)
+		}
+	}
+	vmxData["floppy0.present"] = "FALSE"
+
+	// Rewrite the VMX
+	if err := WriteVMX(vmxPath, vmxData); err != nil {
+		state["error"] = fmt.Errorf("Error writing VMX: %s", err)
+		return multistep.ActionHalt
+	}
+
+	return multistep.ActionContinue
+}
+
+func (stepCleanVMX) Cleanup(map[string]interface{}) {}
+
+func (stepCleanVMX) readVMX(vmxPath string) (map[string]string, error) {
+	vmxF, err := os.Open(vmxPath)
+	if err != nil {
+		return nil, err
+	}
+	defer vmxF.Close()
+
+	vmxBytes, err := ioutil.ReadAll(vmxF)
+	if err != nil {
+		return nil, err
+	}
+
+	return ParseVMX(string(vmxBytes)), nil
+}

builder/vmware/step_compact_disk.go

@@ -0,0 +1,43 @@
+package vmware
+
+import (
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/packer"
+	"log"
+)
+
+// This step compacts the virtual disk for the VM unless the "skip_compaction"
+// boolean is true.
+//
+// Uses:
+//   config *config
+//   driver Driver
+//   full_disk_path string
+//   ui     packer.Ui
+//
+// Produces:
+//   <nothing>
+type stepCompactDisk struct{}
+
+func (stepCompactDisk) Run(state map[string]interface{}) multistep.StepAction {
+	config := state["config"].(*config)
+	driver := state["driver"].(Driver)
+	ui := state["ui"].(packer.Ui)
+	full_disk_path := state["full_disk_path"].(string)
+
+	if config.SkipCompaction == true {
+		log.Println("Skipping disk compaction step...")
+		return multistep.ActionContinue
+	}
+
+	ui.Say("Compacting the disk image")
+	if err := driver.CompactDisk(full_disk_path); err != nil {
+		state["error"] = fmt.Errorf("Error compacting disk: %s", err)
+		return multistep.ActionHalt
+	}
+
+	return multistep.ActionContinue
+}
+
+func (stepCompactDisk) Cleanup(map[string]interface{}) {}

builder/vmware/step_create_disk.go

@@ -15,7 +15,7 @@ import (
 //   ui     packer.Ui
 //
 // Produces:
-//   <nothing>
+//   full_disk_path (string) - The full path to the created disk.
 type stepCreateDisk struct{}
 
 func (stepCreateDisk) Run(state map[string]interface{}) multistep.StepAction {
@@ -24,14 +24,16 @@ func (stepCreateDisk) Run(state map[string]interface{}) multistep.StepAction {
 	ui := state["ui"].(packer.Ui)
 
 	ui.Say("Creating virtual machine disk")
-	output := filepath.Join(config.OutputDir, config.DiskName+".vmdk")
-	if err := driver.CreateDisk(output, fmt.Sprintf("%dM", config.DiskSize)); err != nil {
+	full_disk_path := filepath.Join(config.OutputDir, config.DiskName+".vmdk")
+	if err := driver.CreateDisk(full_disk_path, fmt.Sprintf("%dM", config.DiskSize)); err != nil {
 		err := fmt.Errorf("Error creating disk: %s", err)
 		state["error"] = err
 		ui.Error(err.Error())
 		return multistep.ActionHalt
 	}
 
+	state["full_disk_path"] = full_disk_path
+
 	return multistep.ActionContinue
 }
 

builder/vmware/step_create_vmx.go

@@ -36,10 +36,10 @@ func (stepCreateVMX) Run(state map[string]interface{}) multistep.StepAction {
 	ui.Say("Building and writing VMX file")
 
 	tplData := &vmxTemplateData{
-		config.VMName,
-		config.GuestOSType,
-		config.DiskName,
-		isoPath,
+		Name:     config.VMName,
+		GuestOS:  config.GuestOSType,
+		DiskName: config.DiskName,
+		ISOPath:  isoPath,
 	}
 
 	var buf bytes.Buffer
@@ -55,6 +55,13 @@ func (stepCreateVMX) Run(state map[string]interface{}) multistep.StepAction {
 		}
 	}
 
+	if floppyPathRaw, ok := state["floppy_path"]; ok {
+		log.Println("Floppy path present, setting in VMX")
+		vmxData["floppy0.present"] = "TRUE"
+		vmxData["floppy0.fileType"] = "file"
+		vmxData["floppy0.fileName"] = floppyPathRaw.(string)
+	}
+
 	vmxPath := filepath.Join(config.OutputDir, config.VMName+".vmx")
 	if err := WriteVMX(vmxPath, vmxData); err != nil {
 		err := fmt.Errorf("Error creating VMX file: %s", err)

builder/vmware/step_download_iso.go

@@ -1,7 +1,6 @@
 package vmware
 
 import (
-	"crypto/md5"
 	"encoding/hex"
 	"fmt"
 	"github.com/mitchellh/multistep"
@@ -27,7 +26,7 @@ func (s stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
 	config := state["config"].(*config)
 	ui := state["ui"].(packer.Ui)
 
-	checksum, err := hex.DecodeString(config.ISOMD5)
+	checksum, err := hex.DecodeString(config.ISOChecksum)
 	if err != nil {
 		err := fmt.Errorf("Error parsing checksum: %s", err)
 		state["error"] = err
@@ -43,7 +42,7 @@ func (s stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
 		Url:        config.ISOUrl,
 		TargetPath: cachePath,
 		CopyFile:   false,
-		Hash:       md5.New(),
+		Hash:       common.HashForType(config.ISOChecksumType),
 		Checksum:   checksum,
 	}
 

builder/vmware/step_prepare_output_dir.go

@@ -10,6 +10,12 @@ type stepPrepareOutputDir struct{}
 
 func (stepPrepareOutputDir) Run(state map[string]interface{}) multistep.StepAction {
 	config := state["config"].(*config)
+	ui := state["ui"].(packer.Ui)
+
+	if _, err := os.Stat(config.OutputDir); err == nil && config.PackerForce {
+		ui.Say("Deleting previous output directory...")
+		os.RemoveAll(config.OutputDir)
+	}
 
 	if err := os.MkdirAll(config.OutputDir, 0755); err != nil {
 		state["error"] = err

builder/vmware/step_prepare_tools.go

@@ -0,0 +1,33 @@
+package vmware
+
+import (
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"os"
+)
+
+type stepPrepareTools struct{}
+
+func (*stepPrepareTools) Run(state map[string]interface{}) multistep.StepAction {
+	config := state["config"].(*config)
+	driver := state["driver"].(Driver)
+
+	if config.ToolsUploadFlavor == "" {
+		return multistep.ActionContinue
+	}
+
+	path := driver.ToolsIsoPath(config.ToolsUploadFlavor)
+	if _, err := os.Stat(path); err != nil {
+		state["error"] = fmt.Errorf(
+			"Couldn't find VMware tools for '%s'! VMware often downloads these\n"+
+				"tools on-demand. However, to do this, you need to create a fake VM\n"+
+				"of the proper type then click the 'install tools' option in the\n"+
+				"VMware GUI.", config.ToolsUploadFlavor)
+		return multistep.ActionHalt
+	}
+
+	state["tools_upload_source"] = path
+	return multistep.ActionContinue
+}
+
+func (*stepPrepareTools) Cleanup(map[string]interface{}) {}

builder/vmware/step_provision.go

@@ -1,25 +0,0 @@
-package vmware
-
-import (
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/packer"
-	"log"
-)
-
-type stepProvision struct{}
-
-func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
-	comm := state["communicator"].(packer.Communicator)
-	hook := state["hook"].(packer.Hook)
-	ui := state["ui"].(packer.Ui)
-
-	log.Println("Running the provision hook")
-	if err := hook.Run(packer.HookProvision, ui, comm, nil); err != nil {
-		state["error"] = err
-		return multistep.ActionHalt
-	}
-
-	return multistep.ActionContinue
-}
-
-func (*stepProvision) Cleanup(map[string]interface{}) {}

builder/vmware/step_run.go

@@ -27,13 +27,21 @@ func (s *stepRun) Run(state map[string]interface{}) multistep.StepAction {
 	driver := state["driver"].(Driver)
 	ui := state["ui"].(packer.Ui)
 	vmxPath := state["vmx_path"].(string)
+	vncPort := state["vnc_port"].(uint)
 
 	// Set the VMX path so that we know we started the machine
 	s.bootTime = time.Now()
 	s.vmxPath = vmxPath
 
 	ui.Say("Starting virtual machine...")
-	if err := driver.Start(vmxPath); err != nil {
+	if config.Headless {
+		ui.Message(fmt.Sprintf(
+			"The VM will be run headless, without a GUI. If you want to\n"+
+				"view the screen of the VM, connect via VNC without a password to\n"+
+				"127.0.0.1:%d", vncPort))
+	}
+
+	if err := driver.Start(vmxPath, config.Headless); err != nil {
 		err := fmt.Errorf("Error starting VM: %s", err)
 		state["error"] = err
 		ui.Error(err.Error())
@@ -41,9 +49,9 @@ func (s *stepRun) Run(state map[string]interface{}) multistep.StepAction {
 	}
 
 	// Wait the wait amount
-	if int64(config.BootWait) > 0 {
-		ui.Say(fmt.Sprintf("Waiting %s for boot...", config.BootWait.String()))
-		time.Sleep(config.BootWait)
+	if int64(config.bootWait) > 0 {
+		ui.Say(fmt.Sprintf("Waiting %s for boot...", config.bootWait.String()))
+		time.Sleep(config.bootWait)
 	}
 
 	return multistep.ActionContinue

builder/vmware/step_shutdown.go

@@ -1,11 +1,14 @@
 package vmware
 
 import (
+	"bytes"
 	"errors"
 	"fmt"
 	"github.com/mitchellh/multistep"
 	"github.com/mitchellh/packer/packer"
 	"log"
+	"path/filepath"
+	"strings"
 	"time"
 )
 
@@ -33,7 +36,13 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
 	if config.ShutdownCommand != "" {
 		ui.Say("Gracefully halting virtual machine...")
 		log.Printf("Executing shutdown command: %s", config.ShutdownCommand)
-		cmd := &packer.RemoteCmd{Command: config.ShutdownCommand}
+
+		var stdout, stderr bytes.Buffer
+		cmd := &packer.RemoteCmd{
+			Command: config.ShutdownCommand,
+			Stdout:  &stdout,
+			Stderr:  &stderr,
+		}
 		if err := comm.Start(cmd); err != nil {
 			err := fmt.Errorf("Failed to send shutdown command: %s", err)
 			state["error"] = err
@@ -44,9 +53,20 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
 		// Wait for the command to run
 		cmd.Wait()
 
+		// If the command failed to run, notify the user in some way.
+		if cmd.ExitStatus != 0 {
+			state["error"] = fmt.Errorf(
+				"Shutdown command has non-zero exit status.\n\nStdout: %s\n\nStderr: %s",
+				stdout.String(), stderr.String())
+			return multistep.ActionHalt
+		}
+
+		log.Printf("Shutdown stdout: %s", stdout.String())
+		log.Printf("Shutdown stderr: %s", stderr.String())
+
 		// Wait for the machine to actually shut down
-		log.Printf("Waiting max %s for shutdown to complete", config.ShutdownTimeout)
-		shutdownTimer := time.After(config.ShutdownTimeout)
+		log.Printf("Waiting max %s for shutdown to complete", config.shutdownTimeout)
+		shutdownTimer := time.After(config.shutdownTimeout)
 		for {
 			running, _ := driver.IsRunning(vmxPath)
 			if !running {
@@ -72,6 +92,34 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
 		}
 	}
 
+	ui.Message("Waiting for VMware to clean up after itself...")
+	lockPattern := filepath.Join(config.OutputDir, "*.lck")
+	timer := time.After(15 * time.Second)
+LockWaitLoop:
+	for {
+		locks, err := filepath.Glob(lockPattern)
+		if err == nil {
+			if len(locks) == 0 {
+				log.Println("No more lock files found. VMware is clean.")
+				break
+			}
+
+			if len(locks) == 1 && strings.HasSuffix(locks[0], ".vmx.lck") {
+				log.Println("Only waiting on VMX lock. VMware is clean.")
+				break
+			}
+
+			log.Printf("Waiting on lock files: %#v", locks)
+		}
+
+		select {
+		case <-timer:
+			log.Println("Reached timeout on waiting for clean VMware. Assuming clean.")
+			break LockWaitLoop
+		case <-time.After(1 * time.Second):
+		}
+	}
+
 	log.Println("VM shut down.")
 	return multistep.ActionContinue
 }

builder/vmware/step_type_boot_command.go

@@ -114,6 +114,20 @@ func vncSendString(c *vnc.ClientConn, original string) {
 			continue
 		}
 
+		if strings.HasPrefix(original, "<wait5>") {
+			log.Printf("Special code '<wait5>' found, sleeping 5 seconds")
+			time.Sleep(5 * time.Second)
+			original = original[len("<wait5>"):]
+			continue
+		}
+
+		if strings.HasPrefix(original, "<wait10>") {
+			log.Printf("Special code '<wait10>' found, sleeping 10 seconds")
+			time.Sleep(10 * time.Second)
+			original = original[len("<wait10>"):]
+			continue
+		}
+
 		for specialCode, specialValue := range special {
 			if strings.HasPrefix(original, specialCode) {
 				log.Printf("Special code '%s' found, replacing with: %d", specialCode, specialValue)

builder/vmware/step_upload_tools.go

@@ -0,0 +1,49 @@
+package vmware
+
+import (
+	"bytes"
+	"fmt"
+	"github.com/mitchellh/multistep"
+	"github.com/mitchellh/packer/packer"
+	"os"
+	"text/template"
+)
+
+type toolsUploadPathTemplate struct {
+	Flavor string
+}
+
+type stepUploadTools struct{}
+
+func (*stepUploadTools) Run(state map[string]interface{}) multistep.StepAction {
+	config := state["config"].(*config)
+	if config.ToolsUploadFlavor == "" {
+		return multistep.ActionContinue
+	}
+
+	comm := state["communicator"].(packer.Communicator)
+	tools_source := state["tools_upload_source"].(string)
+	ui := state["ui"].(packer.Ui)
+
+	ui.Say(fmt.Sprintf("Uploading the '%s' VMware Tools", config.ToolsUploadFlavor))
+	f, err := os.Open(tools_source)
+	if err != nil {
+		state["error"] = fmt.Errorf("Error opening VMware Tools ISO: %s", err)
+		return multistep.ActionHalt
+	}
+	defer f.Close()
+
+	tplData := &toolsUploadPathTemplate{Flavor: config.ToolsUploadFlavor}
+	var processedPath bytes.Buffer
+	t := template.Must(template.New("path").Parse(config.ToolsUploadPath))
+	t.Execute(&processedPath, tplData)
+
+	if err := comm.Upload(processedPath.String(), f); err != nil {
+		state["error"] = fmt.Errorf("Error uploading VMware Tools: %s", err)
+		return multistep.ActionHalt
+	}
+
+	return multistep.ActionContinue
+}
+
+func (*stepUploadTools) Cleanup(map[string]interface{}) {}

builder/vmware/step_wait_for_ssh.go

@@ -1,183 +0,0 @@
-package vmware
-
-import (
-	gossh "code.google.com/p/go.crypto/ssh"
-	"errors"
-	"fmt"
-	"github.com/mitchellh/multistep"
-	"github.com/mitchellh/packer/communicator/ssh"
-	"github.com/mitchellh/packer/packer"
-	"io/ioutil"
-	"log"
-	"net"
-	"os"
-	"time"
-)
-
-// This step waits for SSH to become available and establishes an SSH
-// connection.
-//
-// Uses:
-//   config *config
-//   ui     packer.Ui
-//   vmx_path string
-//
-// Produces:
-//   communicator packer.Communicator
-type stepWaitForSSH struct {
-	cancel bool
-	conn   net.Conn
-}
-
-func (s *stepWaitForSSH) Run(state map[string]interface{}) multistep.StepAction {
-	config := state["config"].(*config)
-	ui := state["ui"].(packer.Ui)
-
-	var comm packer.Communicator
-	var err error
-
-	waitDone := make(chan bool, 1)
-	go func() {
-		comm, err = s.waitForSSH(state)
-		waitDone <- true
-	}()
-
-	log.Printf("Waiting for SSH, up to timeout: %s", config.SSHWaitTimeout.String())
-
-	timeout := time.After(config.SSHWaitTimeout)
-WaitLoop:
-	for {
-		// Wait for either SSH to become available, a timeout to occur,
-		// or an interrupt to come through.
-		select {
-		case <-waitDone:
-			if err != nil {
-				ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
-				return multistep.ActionHalt
-			}
-
-			state["communicator"] = comm
-			break WaitLoop
-		case <-timeout:
-			ui.Error("Timeout waiting for SSH.")
-			s.cancel = true
-			return multistep.ActionHalt
-		case <-time.After(1 * time.Second):
-			if _, ok := state[multistep.StateCancelled]; ok {
-				log.Println("Interrupt detected, quitting waiting for SSH.")
-				return multistep.ActionHalt
-			}
-		}
-	}
-
-	return multistep.ActionContinue
-}
-
-func (s *stepWaitForSSH) Cleanup(map[string]interface{}) {
-	if s.conn != nil {
-		s.conn.Close()
-		s.conn = nil
-	}
-}
-
-// Reads the network information for lookup via DHCP.
-func (s *stepWaitForSSH) dhcpLeaseLookup(vmxPath string) (GuestIPFinder, error) {
-	f, err := os.Open(vmxPath)
-	if err != nil {
-		return nil, err
-	}
-	defer f.Close()
-
-	vmxBytes, err := ioutil.ReadAll(f)
-	if err != nil {
-		return nil, err
-	}
-
-	vmxData := ParseVMX(string(vmxBytes))
-
-	var ok bool
-	macAddress := ""
-	if macAddress, ok = vmxData["ethernet0.address"]; !ok || macAddress == "" {
-		if macAddress, ok = vmxData["ethernet0.generatedAddress"]; !ok || macAddress == "" {
-			return nil, errors.New("couldn't find MAC address in VMX")
-		}
-	}
-
-	return &DHCPLeaseGuestLookup{"vmnet8", macAddress}, nil
-}
-
-// This blocks until SSH becomes available, and sends the communicator
-// on the given channel.
-func (s *stepWaitForSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
-	config := state["config"].(*config)
-	ui := state["ui"].(packer.Ui)
-	vmxPath := state["vmx_path"].(string)
-
-	handshakeAttempts := 0
-
-	ui.Say("Waiting for SSH to become available...")
-	var comm packer.Communicator
-	var nc net.Conn
-	for {
-		if nc != nil {
-			nc.Close()
-		}
-
-		time.Sleep(5 * time.Second)
-
-		if s.cancel {
-			log.Println("SSH wait cancelled. Exiting loop.")
-			return nil, errors.New("SSH wait cancelled")
-		}
-
-		// First we wait for the IP to become available...
-		log.Println("Lookup up IP information...")
-		ipLookup, err := s.dhcpLeaseLookup(vmxPath)
-		if err != nil {
-			log.Printf("Can't lookup via DHCP lease: %s", err)
-		}
-
-		ip, err := ipLookup.GuestIP()
-		if err != nil {
-			log.Printf("IP lookup failed: %s", err)
-			continue
-		}
-
-		log.Printf("Detected IP: %s", ip)
-
-		// Attempt to connect to SSH port
-		nc, err = net.Dial("tcp", fmt.Sprintf("%s:%d", ip, config.SSHPort))
-		if err != nil {
-			log.Printf("TCP connection to SSH ip/port failed: %s", err)
-			continue
-		}
-
-		// Then we attempt to connect via SSH
-		sshConfig := &gossh.ClientConfig{
-			User: config.SSHUser,
-			Auth: []gossh.ClientAuth{
-				gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
-			},
-		}
-
-		comm, err = ssh.New(nc, sshConfig)
-		if err != nil {
-			log.Printf("SSH handshake err: %s", err)
-
-			handshakeAttempts += 1
-			if handshakeAttempts < 10 {
-				// Try to connect via SSH a handful of times
-				continue
-			}
-
-			return nil, err
-		}
-
-		ui.Say("Connected via SSH!")
-		break
-	}
-
-	// Store the connection so we can close it later
-	s.conn = nc
-	return comm, nil
-}

command/build/command.go

@@ -21,12 +21,14 @@ func (Command) Help() string {
 
 func (c Command) Run(env packer.Environment, args []string) int {
 	var cfgDebug bool
+	var cfgForce bool
 	var cfgExcept []string
 	var cfgOnly []string
 
 	cmdFlags := flag.NewFlagSet("build", flag.ContinueOnError)
 	cmdFlags.Usage = func() { env.Ui().Say(c.Help()) }
 	cmdFlags.BoolVar(&cfgDebug, "debug", false, "debug mode for builds")
+	cmdFlags.BoolVar(&cfgForce, "force", false, "force a build if artifacts exist")
 	cmdFlags.Var((*stringSliceValue)(&cfgExcept), "except", "build all builds except these")
 	cmdFlags.Var((*stringSliceValue)(&cfgOnly), "only", "only build the given builds by name")
 	if err := cmdFlags.Parse(args); err != nil {
@@ -141,11 +143,13 @@ func (c Command) Run(env packer.Environment, args []string) int {
 	env.Ui().Say("")
 
 	log.Printf("Build debug mode: %v", cfgDebug)
+	log.Printf("Force build: %v", cfgForce)
 
-	// Set the debug mode and prepare all the builds
+	// Set the debug and force mode and prepare all the builds
 	for _, b := range builds {
 		log.Printf("Preparing build: %s", b.Name())
 		b.SetDebug(cfgDebug)
+		b.SetForce(cfgForce)
 		err := b.Prepare()
 		if err != nil {
 			env.Ui().Error(err.Error())

command/build/help.go

@@ -1,7 +1,7 @@
 package build
 
 const helpText = `
-Usage: packer build TEMPLATE
+Usage: packer build [options] TEMPLATE
 
   Will execute multiple builds in parallel as defined in the template.
   The various artifacts created by the template will be outputted.
@@ -9,6 +9,7 @@ Usage: packer build TEMPLATE
 Options:
 
   -debug                     Debug mode enabled for builds
+  -force                     Force a build to continue if artifacts exist, deletes existing artifacts
   -except=foo,bar,baz        Build all builds other than these
   -only=foo,bar,baz          Only build the given builds by name
 `

command/fix/command.go

@@ -0,0 +1,85 @@
+package fix
+
+import (
+	"bytes"
+	"encoding/json"
+	"flag"
+	"fmt"
+	"github.com/mitchellh/packer/packer"
+	"os"
+	"strings"
+)
+
+type Command byte
+
+func (Command) Help() string {
+	return strings.TrimSpace(helpString)
+}
+
+func (c Command) Run(env packer.Environment, args []string) int {
+	cmdFlags := flag.NewFlagSet("fix", flag.ContinueOnError)
+	cmdFlags.Usage = func() { env.Ui().Say(c.Help()) }
+	if err := cmdFlags.Parse(args); err != nil {
+		return 1
+	}
+
+	args = cmdFlags.Args()
+	if len(args) != 1 {
+		cmdFlags.Usage()
+		return 1
+	}
+
+	// Read the file for decoding
+	tplF, err := os.Open(args[0])
+	if err != nil {
+		env.Ui().Error(fmt.Sprintf("Error opening template: %s", err))
+		return 1
+	}
+	defer tplF.Close()
+
+	// Decode the JSON into a generic map structure
+	var templateData map[string]interface{}
+	decoder := json.NewDecoder(tplF)
+	if err := decoder.Decode(&templateData); err != nil {
+		env.Ui().Error(fmt.Sprintf("Error parsing template: %s", err))
+		return 1
+	}
+
+	// Close the file since we're done with that
+	tplF.Close()
+
+	// Run the template through the various fixers
+	fixers := []Fixer{Fixers["iso-md5"]}
+	input := templateData
+	for _, fixer := range fixers {
+		var err error
+		input, err = fixer.Fix(input)
+		if err != nil {
+			env.Ui().Error(fmt.Sprintf("Error fixing: %s", err))
+			return 1
+		}
+	}
+
+	var output bytes.Buffer
+	encoder := json.NewEncoder(&output)
+	if err := encoder.Encode(input); err != nil {
+		env.Ui().Error(fmt.Sprintf("Error encoding: %s", err))
+		return 1
+	}
+
+	var indented bytes.Buffer
+	if err := json.Indent(&indented, output.Bytes(), "", "  "); err != nil {
+		env.Ui().Error(fmt.Sprintf("Error encoding: %s", err))
+		return 1
+	}
+
+	result := indented.String()
+	result = strings.Replace(result, `\u003c`, "<", -1)
+	result = strings.Replace(result, `\u003e`, ">", -1)
+	env.Ui().Say(result)
+	return 0
+}
+
+func (c Command) Synopsis() string {
+	return "fixes templates from old versions of packer"
+}

command/fix/command_test.go

@@ -0,0 +1,14 @@
+package fix
+
+import (
+	"github.com/mitchellh/packer/packer"
+	"testing"
+)
+
+func TestCommand_Impl(t *testing.T) {
+	var raw interface{}
+	raw = new(Command)
+	if _, ok := raw.(packer.Command); !ok {
+		t.Fatalf("must be a Command")
+	}
+}

command/fix/fixer.go

@@ -0,0 +1,18 @@
+package fix
+
+// A Fixer is something that can perform a fix operation on a template.
+type Fixer interface {
+	// Fix takes a raw map structure input, potentially transforms it
+	// in some way, and returns the new, transformed structure. The
+	// Fix method is allowed to mutate the input.
+	Fix(input map[string]interface{}) (map[string]interface{}, error)
+}
+
+// Fixers is the map of all available fixers, by name.
+var Fixers map[string]Fixer
+
+func init() {
+	Fixers = map[string]Fixer{
+		"iso-md5": new(FixerISOMD5),
+	}
+}

command/fix/fixer_iso_md5.go

@@ -0,0 +1,43 @@
+package fix
+
+import (
+	"github.com/mitchellh/mapstructure"
+)
+
+// FixerISOMD5 is a Fixer that replaces the "iso_md5" configuration key
+// with the newer "iso_checksum" and "iso_checksum_type" within builders.
+type FixerISOMD5 struct{}
+
+func (FixerISOMD5) Fix(input map[string]interface{}) (map[string]interface{}, error) {
+	// Our template type we'll use for this fixer only
+	type template struct {
+		Builders []map[string]interface{}
+	}
+
+	// Decode the input into our structure, if we can
+	var tpl template
+	if err := mapstructure.Decode(input, &tpl); err != nil {
+		return nil, err
+	}
+
+	// Go through each builder and replace the iso_md5 if we can
+	for _, builder := range tpl.Builders {
+		md5raw, ok := builder["iso_md5"]
+		if !ok {
+			continue
+		}
+
+		md5, ok := md5raw.(string)
+		if !ok {
+			// TODO: error?
+			continue
+		}
+
+		delete(builder, "iso_md5")
+		builder["iso_checksum"] = md5
+		builder["iso_checksum_type"] = "md5"
+	}
+
+	input["builders"] = tpl.Builders
+	return input, nil
+}

command/fix/fixer_iso_md5_test.go

@@ -0,0 +1,46 @@
+package fix
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestFixerISOMD5_Impl(t *testing.T) {
+	var raw interface{}
+	raw = new(FixerISOMD5)
+	if _, ok := raw.(Fixer); !ok {
+		t.Fatalf("must be a Fixer")
+	}
+}
+
+func TestFixerISOMD5_Fix(t *testing.T) {
+	var f FixerISOMD5
+
+	input := map[string]interface{}{
+		"builders": []interface{}{
+			map[string]string{
+				"type":    "foo",
+				"iso_md5": "bar",
+			},
+		},
+	}
+
+	expected := map[string]interface{}{
+		"builders": []map[string]interface{}{
+			map[string]interface{}{
+				"type":              "foo",
+				"iso_checksum":      "bar",
+				"iso_checksum_type": "md5",
+			},
+		},
+	}
+
+	output, err := f.Fix(input)
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+
+	if !reflect.DeepEqual(output, expected) {
+		t.Fatalf("unexpected: %#v\nexpected: %#v\n", output, expected)
+	}
+}

command/fix/help.go

@@ -0,0 +1,16 @@
+package fix
+
+const helpString = `
+Usage: packer fix [options] TEMPLATE
+
+  Reads the JSON template and attempts to fix known backwards
+  incompatibilities. The fixed template will be outputted to standard out.
+
+  If the template cannot be fixed due to an error, the command will exit
+  with a non-zero exit status. Error messages will appear on standard error.
+
+Fixes that are run:
+
+  iso-md5         Replaces "iso_md5" in builders with newer "iso_checksum"
+
+`

command/validate/help.go

@@ -1,7 +1,7 @@
 package validate
 
 const helpString = `
-Usage: packer validate TEMPLATE
+Usage: packer validate [options] TEMPLATE
 
   Checks the template is valid by parsing the template and also
   checking the configuration with the various builders, provisioners, etc.

communicator/ssh/communicator.go

@@ -3,6 +3,7 @@ package ssh
 import (
 	"bytes"
 	"code.google.com/p/go.crypto/ssh"
+	"errors"
 	"fmt"
 	"github.com/mitchellh/packer/packer"
 	"io"
@@ -13,18 +14,39 @@ import (
 
 type comm struct {
 	client *ssh.ClientConn
+	config *Config
+	conn   net.Conn
+}
+
+// Config is the structure used to configure the SSH communicator.
+type Config struct {
+	// The configuration of the Go SSH connection
+	SSHConfig *ssh.ClientConfig
+
+	// Connection returns a new connection. The current connection
+	// in use will be closed as part of the Close method, or in the
+	// case an error occurs.
+	Connection func() (net.Conn, error)
 }
 
 // Creates a new packer.Communicator implementation over SSH. This takes
 // an already existing TCP connection and SSH configuration.
-func New(c net.Conn, config *ssh.ClientConfig) (result *comm, err error) {
-	client, err := ssh.Client(c, config)
-	result = &comm{client}
+func New(config *Config) (result *comm, err error) {
+	// Establish an initial connection and connect
+	result = &comm{
+		config: config,
+	}
+
+	if err = result.reconnect(); err != nil {
+		result = nil
+		return
+	}
+
 	return
 }
 
 func (c *comm) Start(cmd *packer.RemoteCmd) (err error) {
-	session, err := c.client.NewSession()
+	session, err := c.newSession()
 	if err != nil {
 		return
 	}
@@ -55,7 +77,6 @@ func (c *comm) Start(cmd *packer.RemoteCmd) (err error) {
 	// exit boolean and status.
 	go func() {
 		defer session.Close()
-
 		err := session.Wait()
 		cmd.ExitStatus = 0
 		if err != nil {
@@ -72,8 +93,7 @@ func (c *comm) Start(cmd *packer.RemoteCmd) (err error) {
 }
 
 func (c *comm) Upload(path string, input io.Reader) error {
-	log.Println("Opening new SSH session")
-	session, err := c.client.NewSession()
+	session, err := c.newSession()
 	if err != nil {
 		return err
 	}
@@ -145,6 +165,14 @@ func (c *comm) Upload(path string, input io.Reader) error {
 			// Otherwise, we have an ExitErorr, meaning we can just read
 			// the exit status
 			log.Printf("non-zero exit status: %d", exitErr.ExitStatus())
+
+			// If we exited with status 127, it means SCP isn't available.
+			// Return a more descriptive error for that.
+			if exitErr.ExitStatus() == 127 {
+				return errors.New(
+					"SCP failed to start. This usually means that SCP is not\n" +
+						"properly installed on the remote system.")
+			}
 		}
 
 		return err
@@ -159,3 +187,39 @@ func (c *comm) Upload(path string, input io.Reader) error {
 func (c *comm) Download(string, io.Writer) error {
 	panic("not implemented yet")
 }
+
+func (c *comm) newSession() (*ssh.Session, error) {
+	log.Println("opening new ssh session")
+	session, err := c.client.NewSession()
+	if err != nil {
+		log.Printf("ssh session open error: '%s', attempting reconnect", err)
+		if err := c.reconnect(); err != nil {
+			return nil, err
+		}
+
+		return c.client.NewSession()
+	}
+
+	return session, nil
+}
+
+func (c *comm) reconnect() (err error) {
+	if c.conn != nil {
+		c.conn.Close()
+	}
+
+	log.Printf("reconnecting to TCP connection for SSH")
+	c.conn, err = c.config.Connection()
+	if err != nil {
+		log.Printf("reconnection error: %s", err)
+		return
+	}
+
+	log.Printf("handshaking with SSH")
+	c.client, err = ssh.Client(c.conn, c.config.SSHConfig)
+	if err != nil {
+		log.Printf("handshake error: %s", err)
+	}
+
+	return
+}

communicator/ssh/communicator_test.go

@@ -115,12 +115,20 @@ func TestNew_Invalid(t *testing.T) {
 		},
 	}
 
-	conn, err := net.Dial("tcp", newMockLineServer(t))
-	if err != nil {
-		t.Fatalf("unable to dial to remote side: %s", err)
+	conn := func() (net.Conn, error) {
+		conn, err := net.Dial("tcp", newMockLineServer(t))
+		if err != nil {
+			t.Fatalf("unable to dial to remote side: %s", err)
+		}
+		return conn, err
 	}
 
-	_, err = New(conn, clientConfig)
+	config := &Config{
+		Connection: conn,
+		SSHConfig:  clientConfig,
+	}
+
+	_, err := New(config)
 	if err == nil {
 		t.Fatal("should have had an error connecting")
 	}
@@ -134,12 +142,20 @@ func TestStart(t *testing.T) {
 		},
 	}
 
-	conn, err := net.Dial("tcp", newMockLineServer(t))
-	if err != nil {
-		t.Fatalf("unable to dial to remote side: %s", err)
+	conn := func() (net.Conn, error) {
+		conn, err := net.Dial("tcp", newMockLineServer(t))
+		if err != nil {
+			t.Fatalf("unable to dial to remote side: %s", err)
+		}
+		return conn, err
 	}
 
-	client, err := New(conn, clientConfig)
+	config := &Config{
+		Connection: conn,
+		SSHConfig:  clientConfig,
+	}
+
+	client, err := New(config)
 	if err != nil {
 		t.Fatalf("error connecting to SSH: %s", err)
 	}

communicator/ssh/connect.go

@@ -0,0 +1,33 @@
+package ssh
+
+import (
+	"errors"
+	"log"
+	"net"
+	"time"
+)
+
+// ConnectFunc is a convenience method for returning a function
+// that just uses net.Dial to communicate with the remote end that
+// is suitable for use with the SSH communicator configuration.
+func ConnectFunc(network, addr string, timeout time.Duration) func() (net.Conn, error) {
+	return func() (net.Conn, error) {
+		timeoutCh := time.After(timeout)
+
+		for {
+			select {
+			case <-timeoutCh:
+				return nil, errors.New("timeout connecting to remote machine")
+			default:
+			}
+
+			log.Printf("Opening conn for SSH to %s %s", network, addr)
+			nc, err := net.DialTimeout(network, addr, 15*time.Second)
+			if err == nil {
+				return nc, nil
+			}
+
+			time.Sleep(500 * time.Millisecond)
+		}
+	}
+}

communicator/ssh/password.go

@@ -1,5 +1,7 @@
 package ssh
 
+import "log"
+
 // An implementation of ssh.ClientPassword so that you can use a static
 // string password for the password to ClientAuthPassword.
 type Password string
@@ -7,3 +9,24 @@ type Password string
 func (p Password) Password(user string) (string, error) {
 	return string(p), nil
 }
+
+// An implementation of ssh.ClientKeyboardInteractive that simply sends
+// back the password for all questions. The questions are logged.
+type PasswordKeyboardInteractive string
+
+func (p PasswordKeyboardInteractive) Challenge(user, instruction string, questions []string, echos []bool) ([]string, error) {
+	log.Printf("Keyboard interactive challenge: ")
+	log.Printf("-- User: %s", user)
+	log.Printf("-- Instructions: %s", instruction)
+	for i, question := range questions {
+		log.Printf("-- Question %d: %s", i+1, question)
+	}
+
+	// Just send the password back for all questions
+	answers := make([]string, len(questions))
+	for i, _ := range answers {
+		answers[i] = string(p)
+	}
+
+	return answers, nil
+}

communicator/ssh/password_test.go

@@ -2,6 +2,7 @@ package ssh
 
 import (
 	"code.google.com/p/go.crypto/ssh"
+	"reflect"
 	"testing"
 )
 
@@ -24,3 +25,23 @@ func TestPasswordPassword(t *testing.T) {
 		t.Fatalf("invalid password: %s", result)
 	}
 }
+
+func TestPasswordKeyboardInteractive_Impl(t *testing.T) {
+	var raw interface{}
+	raw = PasswordKeyboardInteractive("foo")
+	if _, ok := raw.(ssh.ClientKeyboardInteractive); !ok {
+		t.Fatal("PasswordKeyboardInteractive must implement ClientKeyboardInteractive")
+	}
+}
+
+func TestPasswordKeybardInteractive_Challenge(t *testing.T) {
+	p := PasswordKeyboardInteractive("foo")
+	result, err := p.Challenge("foo", "bar", []string{"one", "two"}, nil)
+	if err != nil {
+		t.Fatalf("err not nil: %s", err)
+	}
+
+	if !reflect.DeepEqual(result, []string{"foo", "foo"}) {
+		t.Fatalf("invalid password: %#v", result)
+	}
+}

config.go

@@ -27,6 +27,7 @@ const defaultConfig = `
 
 	"commands": {
 		"build": "packer-command-build",
+		"fix": "packer-command-fix",
 		"validate": "packer-command-validate"
 	},
 
@@ -35,6 +36,7 @@ const defaultConfig = `
 	},
 
 	"provisioners": {
+		"file": "packer-provisioner-file",
 		"shell": "packer-provisioner-shell"
 	}
 }

packer.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"github.com/mitchellh/packer/packer"
 	"github.com/mitchellh/packer/packer/plugin"
+	"io"
 	"io/ioutil"
 	"log"
 	"os"
@@ -14,19 +15,37 @@ import (
 )
 
 func main() {
-	if os.Getenv("PACKER_LOG") == "" {
-		// If we don't have logging explicitly enabled, then disable it
-		log.SetOutput(ioutil.Discard)
-	} else {
-		// Logging is enabled, make sure it goes to stderr
-		log.SetOutput(os.Stderr)
+	// Setup logging if PACKER_LOG is set.
+	// Log to PACKER_LOG_PATH if it is set, otherwise default to stderr.
+	var logOutput io.Writer = ioutil.Discard
+	if os.Getenv("PACKER_LOG") != "" {
+		logOutput = os.Stderr
+
+		if logPath := os.Getenv("PACKER_LOG_PATH"); logPath != "" {
+			var err error
+			logOutput, err = os.Create(logPath)
+			if err != nil {
+				fmt.Fprintf(
+					os.Stderr,
+					"Couldn't open '%s' for logging: %s",
+					logPath, err)
+				os.Exit(1)
+			}
+		}
 	}
 
+	log.SetOutput(logOutput)
+
 	// If there is no explicit number of Go threads to use, then set it
 	if os.Getenv("GOMAXPROCS") == "" {
 		runtime.GOMAXPROCS(runtime.NumCPU())
 	}
 
+	log.Printf(
+		"Packer Version: %s %s %s",
+		packer.Version, packer.VersionPrerelease, packer.GitCommit)
+	log.Printf("Packer Target OS/Arch: %s %s", runtime.GOOS, runtime.GOARCH)
+
 	config, err := loadConfig()
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Error loading configuration: \n\n%s\n", err)
@@ -97,7 +116,7 @@ func loadConfig() (*config, error) {
 		mustExist = false
 
 		if err != nil {
-			log.Printf("Error detecing default config file path: %s", err)
+			log.Printf("Error detecting default config file path: %s", err)
 		}
 	}
 

packer/build.go

@@ -6,9 +6,24 @@ import (
 	"sync"
 )
 
-// This is the key in configurations that is set to "true" when Packer
-// debugging is enabled.
-const DebugConfigKey = "packer_debug"
+const (
+	// This is the key in configurations that is set to the name of the
+	// build.
+	BuildNameConfigKey = "packer_build_name"
+
+	// This is the key in the configuration that is set to the type
+	// of the builder that is run. This is useful for provisioners and
+	// such who want to make use of this.
+	BuilderTypeConfigKey = "packer_builder_type"
+
+	// This is the key in configurations that is set to "true" when Packer
+	// debugging is enabled.
+	DebugConfigKey = "packer_debug"
+
+	// This is the key in configurations that is set to "true" when Packer
+	// force build is enabled.
+	ForceConfigKey = "packer_force"
+)
 
 // A Build represents a single job within Packer that is responsible for
 // building some machine image artifact. Builds are meant to be parallelized.
@@ -38,6 +53,12 @@ type Build interface {
 	// When SetDebug is set to true, parallelism between builds is
 	// strictly prohibited.
 	SetDebug(bool)
+
+	// SetForce will enable/disable forcing a build when artifacts exist.
+	//
+	// When SetForce is set to true, existing artifacts from the build are
+	// deleted prior to the build.
+	SetForce(bool)
 }
 
 // A build struct represents a single build job, the result of which should
@@ -54,6 +75,7 @@ type coreBuild struct {
 	provisioners   []coreBuildProvisioner
 
 	debug         bool
+	force         bool
 	l             sync.Mutex
 	prepareCalled bool
 }
@@ -91,12 +113,15 @@ func (b *coreBuild) Prepare() (err error) {
 
 	b.prepareCalled = true
 
-	debugConfig := map[string]interface{}{
-		DebugConfigKey: b.debug,
+	packerConfig := map[string]interface{}{
+		BuildNameConfigKey:   b.name,
+		BuilderTypeConfigKey: b.builderType,
+		DebugConfigKey:       b.debug,
+		ForceConfigKey:       b.force,
 	}
 
 	// Prepare the builder
-	err = b.builder.Prepare(b.builderConfig, debugConfig)
+	err = b.builder.Prepare(b.builderConfig, packerConfig)
 	if err != nil {
 		log.Printf("Build '%s' prepare failure: %s\n", b.name, err)
 		return
@@ -106,7 +131,7 @@ func (b *coreBuild) Prepare() (err error) {
 	for _, coreProv := range b.provisioners {
 		configs := make([]interface{}, len(coreProv.config), len(coreProv.config)+1)
 		copy(configs, coreProv.config)
-		configs = append(configs, debugConfig)
+		configs = append(configs, packerConfig)
 
 		if err = coreProv.provisioner.Prepare(configs...); err != nil {
 			return
@@ -116,7 +141,8 @@ func (b *coreBuild) Prepare() (err error) {
 	// Prepare the post-processors
 	for _, ppSeq := range b.postProcessors {
 		for _, corePP := range ppSeq {
-			if err = corePP.processor.Configure(corePP.config); err != nil {
+			err = corePP.processor.Configure(corePP.config, packerConfig)
+			if err != nil {
 				return
 			}
 		}
@@ -189,7 +215,7 @@ PostProcessorRunSeqLoop:
 			}
 
 			builderUi.Say(fmt.Sprintf("Running post-processor: %s", corePP.processorType))
-			artifact, err := corePP.processor.PostProcess(ppUi, priorArtifact)
+			artifact, keep, err := corePP.processor.PostProcess(ppUi, priorArtifact)
 			if err != nil {
 				errors = append(errors, fmt.Errorf("Post-processor failed: %s", err))
 				continue PostProcessorRunSeqLoop
@@ -200,11 +226,12 @@ PostProcessorRunSeqLoop:
 				continue PostProcessorRunSeqLoop
 			}
 
+			keep = keep || corePP.keepInputArtifact
 			if i == 0 {
 				// This is the first post-processor. We handle deleting
 				// previous artifacts a bit different because multiple
 				// post-processors may be using the original and need it.
-				if !keepOriginalArtifact && corePP.keepInputArtifact {
+				if !keepOriginalArtifact && keep {
 					log.Printf(
 						"Flagging to keep original artifact from post-processor '%s'",
 						corePP.processorType)
@@ -213,7 +240,7 @@ PostProcessorRunSeqLoop:
 			} else {
 				// We have a prior artifact. If we want to keep it, we append
 				// it to the results list. Otherwise, we destroy it.
-				if corePP.keepInputArtifact {
+				if keep {
 					artifacts = append(artifacts, priorArtifact)
 				} else {
 					log.Printf("Deleting prior artifact from post-processor '%s'", corePP.processorType)
@@ -258,6 +285,14 @@ func (b *coreBuild) SetDebug(val bool) {
 	b.debug = val
 }
 
+func (b *coreBuild) SetForce(val bool) {
+	if b.prepareCalled {
+		panic("prepare has already been called")
+	}
+
+	b.force = val
+}
+
 // Cancels the build if it is running.
 func (b *coreBuild) Cancel() {
 	b.builder.Cancel()

packer/build_test.go

@@ -11,6 +11,7 @@ func testBuild() *coreBuild {
 		name:          "test",
 		builder:       &TestBuilder{artifactId: "b"},
 		builderConfig: 42,
+		builderType:   "foo",
 		hooks: map[string][]Hook{
 			"foo": []Hook{&TestHook{}},
 		},
@@ -39,24 +40,29 @@ func TestBuild_Name(t *testing.T) {
 func TestBuild_Prepare(t *testing.T) {
 	assert := asserts.NewTestingAsserts(t, true)
 
-	debugFalseConfig := map[string]interface{}{DebugConfigKey: false}
+	packerConfig := map[string]interface{}{
+		BuildNameConfigKey:   "test",
+		BuilderTypeConfigKey: "foo",
+		DebugConfigKey:       false,
+		ForceConfigKey:       false,
+	}
 
 	build := testBuild()
 	builder := build.builder.(*TestBuilder)
 
 	build.Prepare()
 	assert.True(builder.prepareCalled, "prepare should be called")
-	assert.Equal(builder.prepareConfig, []interface{}{42, debugFalseConfig}, "prepare config should be 42")
+	assert.Equal(builder.prepareConfig, []interface{}{42, packerConfig}, "prepare config should be 42")
 
 	coreProv := build.provisioners[0]
 	prov := coreProv.provisioner.(*TestProvisioner)
 	assert.True(prov.prepCalled, "prepare should be called")
-	assert.Equal(prov.prepConfigs, []interface{}{42, debugFalseConfig}, "prepare should be called with proper config")
+	assert.Equal(prov.prepConfigs, []interface{}{42, packerConfig}, "prepare should be called with proper config")
 
 	corePP := build.postProcessors[0][0]
 	pp := corePP.processor.(*TestPostProcessor)
 	assert.True(pp.configCalled, "config should be called")
-	assert.Equal(pp.configVal, 42, "config should have right value")
+	assert.Equal(pp.configVal, []interface{}{42, packerConfig}, "config should have right value")
 }
 
 func TestBuild_Prepare_Twice(t *testing.T) {
@@ -82,7 +88,12 @@ func TestBuild_Prepare_Twice(t *testing.T) {
 func TestBuild_Prepare_Debug(t *testing.T) {
 	assert := asserts.NewTestingAsserts(t, true)
 
-	debugConfig := map[string]interface{}{DebugConfigKey: true}
+	packerConfig := map[string]interface{}{
+		BuildNameConfigKey:   "test",
+		BuilderTypeConfigKey: "foo",
+		DebugConfigKey:       true,
+		ForceConfigKey:       false,
+	}
 
 	build := testBuild()
 	builder := build.builder.(*TestBuilder)
@@ -90,12 +101,12 @@ func TestBuild_Prepare_Debug(t *testing.T) {
 	build.SetDebug(true)
 	build.Prepare()
 	assert.True(builder.prepareCalled, "prepare should be called")
-	assert.Equal(builder.prepareConfig, []interface{}{42, debugConfig}, "prepare config should be 42")
+	assert.Equal(builder.prepareConfig, []interface{}{42, packerConfig}, "prepare config should be 42")
 
 	coreProv := build.provisioners[0]
 	prov := coreProv.provisioner.(*TestProvisioner)
 	assert.True(prov.prepCalled, "prepare should be called")
-	assert.Equal(prov.prepConfigs, []interface{}{42, debugConfig}, "prepare should be called with proper config")
+	assert.Equal(prov.prepConfigs, []interface{}{42, packerConfig}, "prepare should be called with proper config")
 }
 
 func TestBuild_Run(t *testing.T) {
@@ -239,6 +250,33 @@ func TestBuild_Run_Artifacts(t *testing.T) {
 	if !reflect.DeepEqual(artifactIds, expectedIds) {
 		t.Fatalf("unexpected ids: %#v", artifactIds)
 	}
+
+	// Test case: Test that with a single post-processor that forcibly
+	// keeps inputs, that the artifacts are kept.
+	build = testBuild()
+	build.postProcessors = [][]coreBuildPostProcessor{
+		[]coreBuildPostProcessor{
+			coreBuildPostProcessor{
+				&TestPostProcessor{artifactId: "pp", keep: true}, "pp", 42, false,
+			},
+		},
+	}
+
+	build.Prepare()
+	artifacts, err = build.Run(ui, cache)
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+
+	expectedIds = []string{"b", "pp"}
+	artifactIds = make([]string, len(artifacts))
+	for i, artifact := range artifacts {
+		artifactIds[i] = artifact.Id()
+	}
+
+	if !reflect.DeepEqual(artifactIds, expectedIds) {
+		t.Fatalf("unexpected ids: %#v", artifactIds)
+	}
 }
 
 func TestBuild_RunBeforePrepare(t *testing.T) {

packer/plugin/client.go

@@ -252,7 +252,7 @@ func (c *Client) Start() (address string, err error) {
 	timeout := time.After(c.config.StartTimeout)
 
 	// Start looking for the address
-	log.Println("Waiting for RPC address for: %s", cmd.Path)
+	log.Printf("Waiting for RPC address for: %s", cmd.Path)
 	for done := false; !done; {
 		select {
 		case <-timeout:

packer/plugin/post_processor.go

@@ -10,16 +10,16 @@ type cmdPostProcessor struct {
 	client *Client
 }
 
-func (c *cmdPostProcessor) Configure(config interface{}) error {
+func (c *cmdPostProcessor) Configure(config ...interface{}) error {
 	defer func() {
 		r := recover()
 		c.checkExit(r, nil)
 	}()
 
-	return c.p.Configure(config)
+	return c.p.Configure(config...)
 }
 
-func (c *cmdPostProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.Artifact, error) {
+func (c *cmdPostProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.Artifact, bool, error) {
 	defer func() {
 		r := recover()
 		c.checkExit(r, nil)

packer/plugin/post_processor_test.go

@@ -8,12 +8,12 @@ import (
 
 type helperPostProcessor byte
 
-func (helperPostProcessor) Configure(interface{}) error {
+func (helperPostProcessor) Configure(...interface{}) error {
 	return nil
 }
 
-func (helperPostProcessor) PostProcess(packer.Ui, packer.Artifact) (packer.Artifact, error) {
-	return nil, nil
+func (helperPostProcessor) PostProcess(packer.Ui, packer.Artifact) (packer.Artifact, bool, error) {
+	return nil, false, nil
 }
 
 func TestPostProcessor_NoExist(t *testing.T) {

packer/post_processor.go

@@ -9,9 +9,10 @@ type PostProcessor interface {
 	// Configure is responsible for setting up configuration, storing
 	// the state for later, and returning and errors, such as validation
 	// errors.
-	Configure(interface{}) error
+	Configure(...interface{}) error
 
 	// PostProcess takes a previously created Artifact and produces another
-	// Artifact. If an error occurs, it should return that error.
-	PostProcess(Ui, Artifact) (Artifact, error)
+	// Artifact. If an error occurs, it should return that error. If `keep`
+	// is to true, then the previous artifact is forcibly kept.
+	PostProcess(Ui, Artifact) (a Artifact, keep bool, err error)
 }

packer/post_processor_test.go

@@ -2,22 +2,23 @@ package packer
 
 type TestPostProcessor struct {
 	artifactId   string
+	keep         bool
 	configCalled bool
-	configVal    interface{}
+	configVal    []interface{}
 	ppCalled     bool
 	ppArtifact   Artifact
 	ppUi         Ui
 }
 
-func (pp *TestPostProcessor) Configure(v interface{}) error {
+func (pp *TestPostProcessor) Configure(v ...interface{}) error {
 	pp.configCalled = true
 	pp.configVal = v
 	return nil
 }
 
-func (pp *TestPostProcessor) PostProcess(ui Ui, a Artifact) (Artifact, error) {
+func (pp *TestPostProcessor) PostProcess(ui Ui, a Artifact) (Artifact, bool, error) {
 	pp.ppCalled = true
 	pp.ppArtifact = a
 	pp.ppUi = ui
-	return &TestArtifact{id: pp.artifactId}, nil
+	return &TestArtifact{id: pp.artifactId}, pp.keep, nil
 }

packer/rpc/build.go

@@ -69,6 +69,12 @@ func (b *build) SetDebug(val bool) {
 	}
 }
 
+func (b *build) SetForce(val bool) {
+	if err := b.client.Call("Build.SetForce", val, new(interface{})); err != nil {
+		panic(err)
+	}
+}
+
 func (b *build) Cancel() {
 	if err := b.client.Call("Build.Cancel", new(interface{}), new(interface{})); err != nil {
 		panic(err)
@@ -111,6 +117,11 @@ func (b *BuildServer) SetDebug(val *bool, reply *interface{}) error {
 	return nil
 }
 
+func (b *BuildServer) SetForce(val *bool, reply *interface{}) error {
+	b.build.SetForce(*val)
+	return nil
+}
+
 func (b *BuildServer) Cancel(args *interface{}, reply *interface{}) error {
 	b.build.Cancel()
 	return nil

packer/rpc/build_test.go

@@ -17,6 +17,7 @@ type testBuild struct {
 	runCache       packer.Cache
 	runUi          packer.Ui
 	setDebugCalled bool
+	setForceCalled bool
 	cancelCalled   bool
 
 	errRunResult bool
@@ -48,6 +49,10 @@ func (b *testBuild) SetDebug(bool) {
 	b.setDebugCalled = true
 }
 
+func (b *testBuild) SetForce(bool) {
+	b.setForceCalled = true
+}
+
 func (b *testBuild) Cancel() {
 	b.cancelCalled = true
 }
@@ -104,6 +109,10 @@ func TestBuildRPC(t *testing.T) {
 	bClient.SetDebug(true)
 	assert.True(b.setDebugCalled, "should be called")
 
+	// Test SetForce
+	bClient.SetForce(true)
+	assert.True(b.setForceCalled, "should be called")
+
 	// Test Cancel
 	bClient.Cancel()
 	assert.True(b.cancelCalled, "cancel should be called")

packer/rpc/communicator.go

@@ -153,12 +153,14 @@ func (c *CommunicatorServer) Start(args *CommunicatorStartArgs, reply *interface
 	var cmd packer.RemoteCmd
 	cmd.Command = args.Command
 
+	toClose := make([]net.Conn, 0)
 	if args.StdinAddress != "" {
 		stdinC, err := net.Dial("tcp", args.StdinAddress)
 		if err != nil {
 			return err
 		}
 
+		toClose = append(toClose, stdinC)
 		cmd.Stdin = stdinC
 	}
 
@@ -168,6 +170,7 @@ func (c *CommunicatorServer) Start(args *CommunicatorStartArgs, reply *interface
 			return err
 		}
 
+		toClose = append(toClose, stdoutC)
 		cmd.Stdout = stdoutC
 	}
 
@@ -177,6 +180,7 @@ func (c *CommunicatorServer) Start(args *CommunicatorStartArgs, reply *interface
 			return err
 		}
 
+		toClose = append(toClose, stderrC)
 		cmd.Stderr = stderrC
 	}
 
@@ -196,6 +200,9 @@ func (c *CommunicatorServer) Start(args *CommunicatorStartArgs, reply *interface
 	// exit. When it does, report it back to caller...
 	go func() {
 		defer responseC.Close()
+		for _, conn := range toClose {
+			defer conn.Close()
+		}
 
 		for !cmd.Exited {
 			time.Sleep(50 * time.Millisecond)

packer/rpc/port.go

@@ -20,7 +20,7 @@ func PortRange(min, max int) {
 // bound to that port.
 func netListenerInRange(min, max int) net.Listener {
 	for port := min; port <= max; port++ {
-		l, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
+		l, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", port))
 		if err == nil {
 			return l
 		}

packer/rpc/port_test.go

@@ -16,7 +16,7 @@ func Test_netListenerInRange(t *testing.T) {
 	assert := asserts.NewTestingAsserts(t, true)
 
 	// Open up port 10000 so that we take up a port
-	L1000, err := net.Listen("tcp", ":11000")
+	L1000, err := net.Listen("tcp", "127.0.0.1:11000")
 	defer L1000.Close()
 	assert.Nil(err, "should be able to bind to port 10000")
 

packer/rpc/post_processor.go

@@ -17,50 +17,56 @@ type PostProcessorServer struct {
 	p packer.PostProcessor
 }
 
+type PostProcessorConfigureArgs struct {
+	Configs []interface{}
+}
+
 type PostProcessorProcessResponse struct {
 	Err        error
+	Keep       bool
 	RPCAddress string
 }
 
 func PostProcessor(client *rpc.Client) *postProcessor {
 	return &postProcessor{client}
 }
-func (p *postProcessor) Configure(raw interface{}) (err error) {
-	if cerr := p.client.Call("PostProcessor.Configure", &raw, &err); cerr != nil {
+func (p *postProcessor) Configure(raw ...interface{}) (err error) {
+	args := &PostProcessorConfigureArgs{Configs: raw}
+	if cerr := p.client.Call("PostProcessor.Configure", args, &err); cerr != nil {
 		err = cerr
 	}
 
 	return
 }
 
-func (p *postProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.Artifact, error) {
+func (p *postProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.Artifact, bool, error) {
 	server := rpc.NewServer()
 	RegisterArtifact(server, a)
 	RegisterUi(server, ui)
 
 	var response PostProcessorProcessResponse
 	if err := p.client.Call("PostProcessor.PostProcess", serveSingleConn(server), &response); err != nil {
-		return nil, err
+		return nil, false, err
 	}
 
 	if response.Err != nil {
-		return nil, response.Err
+		return nil, false, response.Err
 	}
 
 	if response.RPCAddress == "" {
-		return nil, nil
+		return nil, false, nil
 	}
 
 	client, err := rpc.Dial("tcp", response.RPCAddress)
 	if err != nil {
-		return nil, err
+		return nil, false, err
 	}
 
-	return Artifact(client), nil
+	return Artifact(client), response.Keep, nil
 }
 
-func (p *PostProcessorServer) Configure(raw *interface{}, reply *error) error {
-	*reply = p.p.Configure(*raw)
+func (p *PostProcessorServer) Configure(args *PostProcessorConfigureArgs, reply *error) error {
+	*reply = p.p.Configure(args.Configs...)
 	if *reply != nil {
 		*reply = NewBasicError(*reply)
 	}
@@ -76,7 +82,7 @@ func (p *PostProcessorServer) PostProcess(address string, reply *PostProcessorPr
 
 	responseAddress := ""
 
-	artifact, err := p.p.PostProcess(&Ui{client}, Artifact(client))
+	artifact, keep, err := p.p.PostProcess(&Ui{client}, Artifact(client))
 	if err == nil && artifact != nil {
 		server := rpc.NewServer()
 		RegisterArtifact(server, artifact)
@@ -89,6 +95,7 @@ func (p *PostProcessorServer) PostProcess(address string, reply *PostProcessorPr
 
 	*reply = PostProcessorProcessResponse{
 		Err:        err,
+		Keep:       keep,
 		RPCAddress: responseAddress,
 	}
 

packer/rpc/post_processor_test.go

@@ -3,6 +3,7 @@ package rpc
 import (
 	"github.com/mitchellh/packer/packer"
 	"net/rpc"
+	"reflect"
 	"testing"
 )
 
@@ -10,23 +11,23 @@ var testPostProcessorArtifact = new(testArtifact)
 
 type TestPostProcessor struct {
 	configCalled bool
-	configVal    interface{}
+	configVal    []interface{}
 	ppCalled     bool
 	ppArtifact   packer.Artifact
 	ppUi         packer.Ui
 }
 
-func (pp *TestPostProcessor) Configure(v interface{}) error {
+func (pp *TestPostProcessor) Configure(v ...interface{}) error {
 	pp.configCalled = true
 	pp.configVal = v
 	return nil
 }
 
-func (pp *TestPostProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.Artifact, error) {
+func (pp *TestPostProcessor) PostProcess(ui packer.Ui, a packer.Artifact) (packer.Artifact, bool, error) {
 	pp.ppCalled = true
 	pp.ppArtifact = a
 	pp.ppUi = ui
-	return testPostProcessorArtifact, nil
+	return testPostProcessorArtifact, false, nil
 }
 
 func TestPostProcessorRPC(t *testing.T) {
@@ -56,14 +57,14 @@ func TestPostProcessorRPC(t *testing.T) {
 		t.Fatal("config should be called")
 	}
 
-	if p.configVal != 42 {
+	if !reflect.DeepEqual(p.configVal, []interface{}{42}) {
 		t.Fatalf("unknown config value: %#v", p.configVal)
 	}
 
 	// Test PostProcess
 	a := new(testArtifact)
 	ui := new(testUi)
-	artifact, err := pClient.PostProcess(ui, a)
+	artifact, _, err := pClient.PostProcess(ui, a)
 	if err != nil {
 		t.Fatalf("err: %s", err)
 	}

packer/template.go

@@ -1,9 +1,11 @@
 package packer
 
 import (
+	"bytes"
 	"encoding/json"
 	"fmt"
 	"github.com/mitchellh/mapstructure"
+	"sort"
 )
 
 // The rawTemplate struct represents the structure of a template read
@@ -14,7 +16,7 @@ type rawTemplate struct {
 	Builders       []map[string]interface{}
 	Hooks          map[string][]string
 	Provisioners   []map[string]interface{}
-	PostProcessors []interface{} `json:"post-processors"`
+	PostProcessors []interface{} `mapstructure:"post-processors"`
 }
 
 // The Template struct represents a parsed template, parsed into the most
@@ -62,20 +64,70 @@ type rawProvisionerConfig struct {
 // and checking for this can be useful, if you wish to format it in a certain
 // way.
 func ParseTemplate(data []byte) (t *Template, err error) {
+	var rawTplInterface interface{}
+	err = json.Unmarshal(data, &rawTplInterface)
+	if err != nil {
+		syntaxErr, ok := err.(*json.SyntaxError)
+		if !ok {
+			return
+		}
+
+		// We have a syntax error. Extract out the line number and friends.
+		// https://groups.google.com/forum/#!topic/golang-nuts/fizimmXtVfc
+		newline := []byte{'\x0a'}
+
+		// Calculate the start/end position of the line where the error is
+		start := bytes.LastIndex(data[:syntaxErr.Offset], newline) + 1
+		end := len(data)
+		if idx := bytes.Index(data[start:], newline); idx >= 0 {
+			end = start + idx
+		}
+
+		// Count the line number we're on plus the offset in the line
+		line := bytes.Count(data[:start], newline) + 1
+		pos := int(syntaxErr.Offset) - start - 1
+
+		err = fmt.Errorf("Error in line %d, char %d: %s\n%s",
+			line, pos, syntaxErr, data[start:end])
+
+		return
+	}
+
+	// Decode the raw template interface into the actual rawTemplate
+	// structure, checking for any extranneous keys along the way.
+	var md mapstructure.Metadata
 	var rawTpl rawTemplate
-	err = json.Unmarshal(data, &rawTpl)
+	decoderConfig := &mapstructure.DecoderConfig{
+		Metadata: &md,
+		Result:   &rawTpl,
+	}
+
+	decoder, err := mapstructure.NewDecoder(decoderConfig)
 	if err != nil {
 		return
 	}
 
+	err = decoder.Decode(rawTplInterface)
+	if err != nil {
+		return
+	}
+
+	errors := make([]error, 0)
+
+	if len(md.Unused) > 0 {
+		sort.Strings(md.Unused)
+		for _, unused := range md.Unused {
+			errors = append(
+				errors, fmt.Errorf("Unknown root level key in template: '%s'", unused))
+		}
+	}
+
 	t = &Template{}
 	t.Builders = make(map[string]rawBuilderConfig)
 	t.Hooks = rawTpl.Hooks
 	t.PostProcessors = make([][]rawPostProcessorConfig, len(rawTpl.PostProcessors))
 	t.Provisioners = make([]rawProvisionerConfig, len(rawTpl.Provisioners))
 
-	errors := make([]error, 0)
-
 	// Gather all the builders
 	for i, v := range rawTpl.Builders {
 		var raw rawBuilderConfig
@@ -169,6 +221,11 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 			continue
 		}
 
+		// The provisioners not only don't need or want the override settings
+		// (as they are processed as part of the preparation below), but will
+		// actively reject them as invalid configuration.
+		delete(v, "override")
+
 		raw.rawConfig = v
 	}
 
@@ -179,6 +236,7 @@ func ParseTemplate(data []byte) (t *Template, err error) {
 	// If there were errors, we put it into a MultiError and return
 	if len(errors) > 0 {
 		err = &MultiError{errors}
+		t = nil
 		return
 	}
 

packer/template_test.go

@@ -2,6 +2,7 @@ package packer
 
 import (
 	"cgl.tideland.biz/asserts"
+	"reflect"
 	"sort"
 	"testing"
 )
@@ -37,6 +38,23 @@ func TestParseTemplate_Invalid(t *testing.T) {
 	assert.Nil(result, "should have no result")
 }
 
+func TestParseTemplate_InvalidKeys(t *testing.T) {
+	assert := asserts.NewTestingAsserts(t, true)
+
+	// Note there is an extra comma below for a purposeful
+	// syntax error in the JSON.
+	data := `
+	{
+		"builders": [{"type": "foo"}],
+		"what is this": ""
+	}
+	`
+
+	result, err := ParseTemplate([]byte(data))
+	assert.NotNil(err, "should have an error")
+	assert.Nil(result, "should have no result")
+}
+
 func TestParseTemplate_BuilderWithoutType(t *testing.T) {
 	assert := asserts.NewTestingAsserts(t, true)
 
@@ -539,7 +557,22 @@ func TestTemplate_Build_ProvisionerOverride(t *testing.T) {
 	`
 
 	template, err := ParseTemplate([]byte(data))
-	assert.Nil(err, "should not error")
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
+
+	rawConfig := template.Provisioners[0].rawConfig
+	if rawConfig == nil {
+		t.Fatal("missing provisioner raw config")
+	}
+
+	expected := map[string]interface{}{
+		"type": "test-prov",
+	}
+
+	if !reflect.DeepEqual(rawConfig, expected) {
+		t.Fatalf("bad raw: %#v", rawConfig)
+	}
 
 	builder := testBuilder()
 	builderMap := map[string]Builder{