Compare commits

..

95 Commits

Author SHA1 Message Date
Mitchell Hashimoto bdc44a53ab v0.2.0 2013-07-16 11:26:02 -07:00
Mitchell Hashimoto c668ef8f70 builder/common: only count handshake error if auth tried 2013-07-16 11:15:47 -07:00
Mitchell Hashimoto 55882a65c7 builder/vmware: error if guest IP is blank [GH-189] 2013-07-16 11:05:47 -07:00
Mitchell Hashimoto 28a22a5fbe builder/*: convert to common StepProvision 2013-07-16 15:44:41 +09:00
Mitchell Hashimoto 55d3f87f3f builder/common: add StepProvision 2013-07-16 15:43:01 +09:00
Mitchell Hashimoto 2cea79c54a builder/amazon/ebs: rename to amazon/ebs in order to support others 2013-07-15 15:02:18 +09:00
Mitchell Hashimoto 3d44633712 Merge branch 'common-ssh'
This unifies all the builders so that they all wait for SSH using a common
step available in builder/common. This common step has the best practices of
every previously unique step, such as retrying SSH handshakes, limiting
the number of SSH handshakes, timeouts, and so on.
2013-07-15 14:23:08 +09:00
Mitchell Hashimoto 61f02c5d02 CHANGELOG 2013-07-15 14:22:55 +09:00
Mitchell Hashimoto d49480ebff builder/vmware: convert to cmmon ssh step 2013-07-15 14:22:13 +09:00
Mitchell Hashimoto 60b8e1f2dc builder/virtualbox: use common SSH connect step 2013-07-15 14:17:09 +09:00
Mitchell Hashimoto d3ea6c96c5 builder/digitalocean: use common connect ssh 2013-07-15 14:14:23 +09:00
Mitchell Hashimoto 2ee81f1491 builder/common: if cancel during SSH, cancel the attempts 2013-07-15 14:10:30 +09:00
Mitchell Hashimoto 51e998a81f builder/amazonebs: switch to common SSH connect step 2013-07-15 14:06:41 +09:00
Mitchell Hashimoto 2f7097bcfe builder/common: add common StepConnectSSH for builders 2013-07-15 11:41:48 +09:00
Mitchell Hashimoto 5aa8448576 website: document new shell env vars 2013-07-15 10:14:36 +09:00
Mitchell Hashimoto d8a0128d1f provisioner/shell: make build name and builder type env vars [GH-154] 2013-07-15 10:07:49 +09:00
Mitchell Hashimoto b1c7d93ee8 packer: Make builder type available in configs [GH-154] 2013-07-15 09:58:32 +09:00
Mitchell Hashimoto a47ad137a0 packer: move constants into a single const() 2013-07-15 09:55:41 +09:00
Mitchell Hashimoto 85f92e35f6 Merge pull request #181 from qur/overrides
packer: Delete "override" key for config passed to provisioners
2013-07-14 17:22:01 -07:00
Mitchell Hashimoto 2895523e45 packer: Add failing test for [GH-181] to fix 2013-07-15 09:21:19 +09:00
Julian Phillips 37f78fe796 Don't expose overrides directly to provisioners
The overrides are processed, so that the provisioner only sees the
appropriate sub-settings, so the provisioner does not need the raw
overrides.

If we leave the top-level overrides object in the provisioner
configuration, then it will be rejected as an unknown configuration
key.
2013-07-14 22:10:20 +01:00
Mitchell Hashimoto 2ba0876d34 builder/vmware: unexport calculted config fields 2013-07-14 21:23:46 +09:00
Mitchell Hashimoto 033c90cfa3 builder/virtualbox: unexport calculated fields 2013-07-14 21:22:11 +09:00
Mitchell Hashimoto 1a6fca7563 builder/digitalocean: unexport calcaulted settings 2013-07-14 21:20:29 +09:00
Mitchell Hashimoto 039a5019e4 builder/amazonebs: Don't export sshTimeout so it can't be set 2013-07-14 21:18:18 +09:00
Mitchell Hashimoto b353c4946e communicator/ssh: improved logging 2013-07-14 21:07:32 +09:00
Mitchell Hashimoto 544a8de3e8 Update CHANGELOG 2013-07-14 21:06:19 +09:00
Mitchell Hashimoto 559e0a7d6f communicator/ssh: tweakable timeout on retry connection 2013-07-14 21:05:22 +09:00
Mitchell Hashimoto 21113fd550 communicator/ssh: retry connection in ConnectFunc forawhile 2013-07-14 21:02:47 +09:00
Mitchell Hashimoto 343614a172 fmt 2013-07-14 20:55:27 +09:00
Mitchell Hashimoto f65711c716 communicator/ssh: re-establish ssh connection if possible [GH-152] 2013-07-14 20:55:02 +09:00
Mitchell Hashimoto 076a4befab communicator/ssh: ConnectFunc times out after 15 sceonds 2013-07-14 20:31:51 +09:00
Mitchell Hashimoto 4d297315c4 communicator/ssh: have a Connection func so we can re-establish
[GH-152]
2013-07-14 20:22:57 +09:00
Mitchell Hashimoto 89e12245af website: document the fix command 2013-07-14 17:38:31 +09:00
Mitchell Hashimoto 06186c97b6 website: document the checksum features 2013-07-14 17:28:53 +09:00
Mitchell Hashimoto 46f37ab8e3 Update CHANGELOG for packer fix 2013-07-14 17:13:19 +09:00
Mitchell Hashimoto e5773e1b66 command/fix: properly replace the escaped </> 2013-07-14 17:10:50 +09:00
Mitchell Hashimoto 6f86fd9849 command/fix: run the iso-md5 fixer 2013-07-14 17:05:26 +09:00
Mitchell Hashimoto f91d4f2244 command/fix: fixer for iso_md5 change 2013-07-14 16:51:20 +09:00
Mitchell Hashimoto 8ee8612997 command/fix: boilerplate for the fix command 2013-07-14 16:22:18 +09:00
Mitchell Hashimoto 82915de7f9 builder/common: support sha1/sha256 hashes 2013-07-14 16:13:07 +09:00
Mitchell Hashimoto eeffebc3e2 Update CHANGELOG 2013-07-14 16:07:08 +09:00
Mitchell Hashimoto 5b4fc718aa builder/vmware: generic checksum types [GH-175] 2013-07-14 15:58:56 +09:00
Mitchell Hashimoto 5a529ee41a builder/virtualbox: support generic hash types [GH-175] 2013-07-14 15:50:34 +09:00
Mitchell Hashimoto 4f4af1621f Update CHANGELOG 2013-07-14 12:30:09 +09:00
Mark Peek 839045264e Move AWS auth decisions to goamz [GH-160]
Currently the passed in AWS auth or AWS environment variables are
interpreted by packer. This change moves that logic into goamz in
order to support both the existing and instance based IAM role
authentication. This requires a corresponding change to goamz.
2013-07-14 12:29:43 +09:00
Mitchell Hashimoto da45a790e1 website: add PACKER_LOG_PATH docs 2013-07-14 11:20:31 +09:00
Mitchell Hashimoto 8e79bb44a7 Clean up some of the log path detection code
/cc @markpeek I think this is much cleaner. Also, I made it so it
doesn't append by default... I like the idea of appending but it scares
me that for a bug report someone might put like a multi-megabyte gist of
logs at me. HAHA. We'll see.
2013-07-14 11:00:44 +09:00
Mitchell Hashimoto 097e44da48 Merge pull request #168 from markpeek/markpeek-filelogging
Add ability to send log output to a file using PACKER_LOG_PATH
2013-07-13 18:55:12 -07:00
Mitchell Hashimoto b358a0ea6e Error if unknown root level key in template [GH-180] 2013-07-14 10:29:14 +09:00
Mitchell Hashimoto a354f5fc08 Unused keys are invalid in templates [GH-104] 2013-07-14 09:28:56 +09:00
Mark Peek bad2bfc4e5 Add PACKER_LOG_PATH for logging to a file
When the environment variables PACKER_LOG and PACKER_LOG_PATH are both
set the log output will be appended to the PACKER_LOG_PATH file.
2013-07-13 15:51:27 +00:00
Mitchell Hashimoto 39b3fe794b buidler/virtualbox,vmware: only stat if not force
/cc @jasonberanek - Swapped this because why even bother doing a stat
if we're forcing anyways.
2013-07-13 14:47:08 +09:00
Mitchell Hashimoto 96861cf216 Merge pull request #179 from jasonberanek/force-build
builder/virtualbox, vmware: Remove output directory in a step when "-force" flag step [GH-178]
2013-07-12 22:45:42 -07:00
Jason A. Beranek 442c5e3e3b Remove output directory in a step when "-force" flag step [GH-178] 2013-07-12 21:23:00 -05:00
Mitchell Hashimoto 39ec791bb2 Merge pull request #176 from smerrill/better-contributor-notes
Add instructions on how to get Go set up to contribute.
2013-07-12 18:04:59 -07:00
Mitchell Hashimoto 7c2f50d4dc fmt 2013-07-13 10:01:27 +09:00
Mitchell Hashimoto e8082e3c72 Update CHANGELOG 2013-07-13 10:01:19 +09:00
Mitchell Hashimoto f131b622be Merge pull request #173 from jasonberanek/force-build
packer, builder/*: Add support for -force flag on builds [GH-119]
2013-07-12 18:00:21 -07:00
Steven Merrill db5b98f79b Add instructions for how to get Go set up to contribute. 2013-07-12 14:40:03 -04:00
Mitchell Hashimoto b33c914fea website: add docs for new security_group_id 2013-07-12 17:27:03 +09:00
Mitchell Hashimoto edb9534773 Merge pull request #162 from brendanhay/persistent-security-group
builder/amazonebs: Ability to specify a pre-existing SecurityGroupId for the AmazonEBS Builder
2013-07-12 01:19:25 -07:00
Mitchell Hashimoto 65a0042a0d Update CHANGELOG 2013-07-12 17:01:38 +09:00
Mitchell Hashimoto 4184926cc8 Merge pull request #164 from mitchellh/do-env-vars
builder/digitalocean: use detected env variables for credentials
2013-07-12 01:00:50 -07:00
Jack Pearkes bc4704f5ca builder/digitalocean: add tests for credentials via env vars 2013-07-12 09:47:45 +02:00
Jason A. Beranek 71a1bd7354 Add support for -force flag on builds [GH-119] 2013-07-11 23:43:23 -05:00
Mitchell Hashimoto 0211c033f4 fmt 2013-07-12 07:01:30 +09:00
Mitchell Hashimoto c88f75b234 Update CHANGELOG 2013-07-12 07:01:23 +09:00
Mitchell Hashimoto 1218c11776 Merge pull request #172 from MattSurabian/feature/ebs-builder-bug
builder/amazonebs: Validate that instance exists and is EBS based before trying to launch it
2013-07-11 14:59:02 -07:00
Matt Surabian f9decb7aca Validate that instance exists and is EBS based before trying to launch it. 2013-07-11 17:41:17 -04:00
Mitchell Hashimoto 8d2e32ea3b Merge pull request #170 from markpeek/markpeek-awsinstancedebugging
builder/amazonebs: Add AWS instance id to normal output
2013-07-11 13:32:49 -07:00
Mark Peek 7f5baf9ca4 Add AWS instance id to normal output
The AWS instance id is only output when logging is occurring. If an
error happens it is useful to know the AWS instance id of the instance
spun up by packer in case further cleanup or forensics are needed.
2013-07-11 20:27:44 +00:00
Mark Peek fb4a86ef35 Add ability to send log output to a file
Using PACKER_LOG=1 causes all the log output to be sent to Stderr. This
change maintains that backward compatility. Anything other than "1" will
be treated as a filename which will have logging appended to that file.
This is useful, for example, to always have debugging available without
cluttering up stdout (and without having to redirect stderr all the time).
2013-07-11 19:42:30 +00:00
Jack Pearkes 3461da5411 builder/digitalocean: use detected env variables for credentials 2013-07-11 11:31:09 +02:00
Brendan Hay fdc126afa7 Adding the ability to specify a security_group_id for the amazonebs builder 2013-07-10 09:32:50 +02:00
Mitchell Hashimoto d9c77c66f4 builder/vmware: error if shutdown command failed 2013-07-09 14:39:02 -07:00
Mitchell Hashimoto 8537d08caa scripts: add -e to build.sh again 2013-07-09 12:42:54 -07:00
Mitchell Hashimoto a5e5e689e0 Update CHANGELOG 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto 86ec808fe0 builder/virtualbox: paused is still running 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto e60945ed7b builder/virtualbox: remove floppy drive before exporting 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto c9dc998b0e website: document the floppy_files feature 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto efc9e9db6e builder/virtualbox: support floppy_files 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto eaa6c9fec1 packer: Only trim whitespace on the right of prefixed UI 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto 65b00775e8 builder/common: continue after creating floppy 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto 24fad7d443 script: build completes 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto cae89a49a1 builder/vmware: support floppy_files for mounting a floppy disk 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto 70f63492f9 builder/common: step to create floppy disks 2013-07-09 12:41:36 -07:00
Mitchell Hashimoto 13ed6c53b2 Merge pull request #157 from rvoicilas/master
Fix typo in packer.go
2013-07-08 23:37:18 -07:00
Radu Voicilas bd937ec8f2 Fix typo 2013-07-09 09:28:07 +03:00
Steven Merrill c80b1bb75d Don't allow a dirty Git tree to fail the make command. 2013-07-09 01:24:19 -04:00
Mitchell Hashimoto 207c15b91b packer: only show git commit in version output for prerelease 2013-07-08 15:38:14 -07:00
Mitchell Hashimoto 3b32ba9f11 scripts: Add git commit to builds 2013-07-08 15:37:01 -07:00
Mitchell Hashimoto d4e8321a80 Up version for dev 2013-07-08 15:17:23 -07:00
Mitchell Hashimoto 00d04d68d9 Merge pull request #150 from dregin/master
website: Just fixing a small error in the text.
2013-07-08 08:47:09 -07:00
Bernard McKeever 845349b5a9 Making sense of the sentence
The sentence was missing a few words.
I've made it read similar to the previous 3 sentences.
2013-07-08 11:09:16 +01:00
92 changed files with 2169 additions and 1040 deletions
+49
View File
@@ -1,3 +1,52 @@
## 0.2.0 (July 16, 2013)
BACKWARDS INCOMPATIBILITIES:
* "iso_md5" in the virtualbox and vmware builders is replaced with
"iso_checksum" and "iso_checksum_type" (with the latter set to "md5").
See the announce below on `packer fix` to automatically fix your templates.
FEATURES:
* **NEW COMMAND:** `packer fix` will attempt to fix templates from older
versions of Packer that are now broken due to backwards incompatibilities.
This command will fix the backwards incompatibilities introduced in this
version.
* Amazon EBS builder can now optionally use a pre-made security group
instead of randomly generating one.
* DigitalOcean API key and client IDs can now be passed in as
environmental variables. See the documentatin for more details.
* VirtualBox and VMware can now have `floppy_files` specified to attach
floppy disks when booting. This allows for unattended Windows installs.
* `packer build` has a new `-force` flag that forces the removal of
existing artifacts if they exist. [GH-173]
* You can now log to a file (instead of just stderr) by setting the
`PACKER_LOG_FILE` environmental variable. [GH-168]
* Checksums other than MD5 can now be used. SHA1 and SHA256 can also
be used. See the documentation on `iso_checksum_type` for more info. [GH-175]
IMPROVEMENTS:
* core: invalid keys in configuration are now considered validation
errors. [GH-104]
* core: all builders now share a common SSH connection core, improving
SSH reliability over all the builders.
* amazon-ebs: Credentials will come from IAM role if available. [GH-160]
* amazon-ebs: Verify the source AMI is EBS-backed before launching. [GH-169]
* shell provisioner: the build name and builder type are available in
the `PACKER_BUILD_NAME` and `PACKER_BUILDER_TYPE` env vars by default,
respectively. [GH-154]
* vmware: error if shutdown command has non-zero exit status.
BUG FIXES:
* core: UI messages are now properly prefixed with spaces again.
* core: If SSH connection ends, re-connection attempts will take
place. [GH-152]
* virtualbox: "paused" doesn't mean the VM is stopped, improving
shutdown detection.
* vmware: error if guest IP could not be detected. [GH-189]
## 0.1.5 (July 7, 2013)
FEATURES:
+24
View File
@@ -41,3 +41,27 @@ it raises the chances we can quickly merge or address your contributions.
linked.
5. The issue is closed.
## Setting up Go to work on Packer
If you have never worked with Go before, you will have to complete the
following steps in order to be able to compile and test Packer.
1. Install Go. On a Mac, you can `brew install go`.
2. Set and export the `GOPATH` environment variable. For example, you can
add `export GOPATH=$HOME/Documents/golang` to your `.bash_profile`.
3. Download the Packer source (and its dependencies) by running
`go get github.com/mitchellh/packer`. This will download the Packer
source to `$GOPATH/src/github.com/mitchellh/packer`.
4. Make your changes to the Packer source. You can run `make` from the main
source directory to recompile all the binaries. Any compilation errors
will be shown when the binaries are rebuilding.
5. Test your changes by running `make test` and then running
`$GOPATH/src/github.com/mitchellh/packer/bin/packer` to build a machine.
6. If everything works well and the tests pass, run `go fmt` on your code
before submitting a pull request.
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"fmt"
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"cgl.tideland.biz/asserts"
@@ -3,7 +3,7 @@
//
// In general, there are two types of AMIs that can be created: ebs-backed or
// instance-store. This builder _only_ builds ebs-backed images.
package amazonebs
package ebs
import (
"errors"
@@ -15,7 +15,8 @@ import (
"github.com/mitchellh/packer/builder/common"
"github.com/mitchellh/packer/packer"
"log"
"os"
"sort"
"strings"
"text/template"
"time"
)
@@ -29,18 +30,21 @@ type config struct {
SecretKey string `mapstructure:"secret_key"`
// Information for the source instance
Region string
SourceAmi string `mapstructure:"source_ami"`
InstanceType string `mapstructure:"instance_type"`
SSHUsername string `mapstructure:"ssh_username"`
SSHPort int `mapstructure:"ssh_port"`
SSHTimeout time.Duration
Region string
SourceAmi string `mapstructure:"source_ami"`
InstanceType string `mapstructure:"instance_type"`
SSHUsername string `mapstructure:"ssh_username"`
SSHPort int `mapstructure:"ssh_port"`
SecurityGroupId string `mapstructure:"security_group_id"`
// Configuration of the resulting AMI
AMIName string `mapstructure:"ami_name"`
PackerDebug bool `mapstructure:"packer_debug"`
RawSSHTimeout string `mapstructure:"ssh_timeout"`
// Unexported fields that are calculated from others
sshTimeout time.Duration
}
type Builder struct {
@@ -49,29 +53,36 @@ type Builder struct {
}
func (b *Builder) Prepare(raws ...interface{}) error {
var err error
var md mapstructure.Metadata
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &b.config,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return err
}
for _, raw := range raws {
err := mapstructure.Decode(raw, &b.config)
err := decoder.Decode(raw)
if err != nil {
return err
}
}
if b.config.AccessKey == "" {
b.config.AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
}
// Accumulate any errors
errs := make([]error, 0)
if b.config.AccessKey == "" {
b.config.AccessKey = os.Getenv("AWS_ACCESS_KEY")
}
if b.config.SecretKey == "" {
b.config.SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
}
if b.config.SecretKey == "" {
b.config.SecretKey = os.Getenv("AWS_SECRET_KEY")
// Unused keys are errors
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
if unused != "type" && !strings.HasPrefix(unused, "packer_") {
errs = append(
errs, fmt.Errorf("Unknown configuration key: %s", unused))
}
}
}
if b.config.SSHPort == 0 {
@@ -83,16 +94,6 @@ func (b *Builder) Prepare(raws ...interface{}) error {
}
// Accumulate any errors
errs := make([]error, 0)
if b.config.AccessKey == "" {
errs = append(errs, errors.New("An access_key must be specified"))
}
if b.config.SecretKey == "" {
errs = append(errs, errors.New("A secret_key must be specified"))
}
if b.config.SourceAmi == "" {
errs = append(errs, errors.New("A source_ami must be specified"))
}
@@ -111,7 +112,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
errs = append(errs, errors.New("An ssh_username must be specified"))
}
b.config.SSHTimeout, err = time.ParseDuration(b.config.RawSSHTimeout)
b.config.sshTimeout, err = time.ParseDuration(b.config.RawSSHTimeout)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing ssh_timeout: %s", err))
}
@@ -139,7 +140,11 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
panic("region not found")
}
auth := aws.Auth{b.config.AccessKey, b.config.SecretKey}
auth, err := aws.GetAuth(b.config.AccessKey, b.config.SecretKey)
if err != nil {
return nil, err
}
ec2conn := ec2.New(auth, region)
// Setup the state bag and initial state for the steps
@@ -154,8 +159,12 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
&stepKeyPair{},
&stepSecurityGroup{},
&stepRunSourceInstance{},
&stepConnectSSH{},
&stepProvision{},
&common.StepConnectSSH{
SSHAddress: sshAddress,
SSHConfig: sshConfig,
SSHWaitTimeout: b.config.sshTimeout,
},
&common.StepProvision{},
&stepStopInstance{},
&stepCreateAMI{},
}
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"github.com/mitchellh/packer/packer"
@@ -47,55 +47,6 @@ func TestBuilder_Prepare_BadType(t *testing.T) {
}
}
func TestBuilderPrepare_AccessKey(t *testing.T) {
var b Builder
config := testConfig()
// Test good
config["access_key"] = "foo"
err := b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.AccessKey != "foo" {
t.Errorf("access key invalid: %s", b.config.AccessKey)
}
// Test bad
delete(config, "access_key")
b = Builder{}
err = b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test env
delete(config, "access_key")
os.Setenv("AWS_ACCESS_KEY_ID", "foo")
defer os.Setenv("AWS_ACCESS_KEY_ID", "")
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.AccessKey != "foo" {
t.Errorf("access key invalid: %s", b.config.AccessKey)
}
delete(config, "access_key")
os.Setenv("AWS_ACCESS_KEY", "foo")
defer os.Setenv("AWS_ACCESS_KEY", "")
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.AccessKey != "foo" {
t.Errorf("access key invalid: %s", b.config.AccessKey)
}
}
func TestBuilderPrepare_AMIName(t *testing.T) {
var b Builder
config := testConfig()
@@ -148,6 +99,18 @@ func TestBuilderPrepare_InstanceType(t *testing.T) {
}
}
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_Region(t *testing.T) {
var b Builder
config := testConfig()
@@ -180,55 +143,6 @@ func TestBuilderPrepare_Region(t *testing.T) {
}
}
func TestBuilderPrepare_SecretKey(t *testing.T) {
var b Builder
config := testConfig()
// Test good
config["secret_key"] = "foo"
err := b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.SecretKey != "foo" {
t.Errorf("secret key invalid: %s", b.config.SecretKey)
}
// Test bad
delete(config, "secret_key")
b = Builder{}
err = b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test env
delete(config, "secret_key")
os.Setenv("AWS_SECRET_ACCESS_KEY", "foo")
defer os.Setenv("AWS_SECRET_ACCESS_KEY", "")
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.SecretKey != "foo" {
t.Errorf("access key invalid: %s", b.config.SecretKey)
}
delete(config, "secret_key")
os.Setenv("AWS_SECRET_KEY", "foo")
defer os.Setenv("AWS_SECRET_KEY", "")
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.SecretKey != "foo" {
t.Errorf("access key invalid: %s", b.config.SecretKey)
}
}
func TestBuilderPrepare_SourceAmi(t *testing.T) {
var b Builder
config := testConfig()
@@ -1,4 +1,4 @@
package amazonebs
package ebs
// This hook is fired prior to launching the EC2 instance.
const HookPreLaunch = "amazonebs_pre_launch"
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"fmt"
+31
View File
@@ -0,0 +1,31 @@
package ebs
import (
gossh "code.google.com/p/go.crypto/ssh"
"fmt"
"github.com/mitchellh/goamz/ec2"
"github.com/mitchellh/packer/communicator/ssh"
)
func sshAddress(state map[string]interface{}) (string, error) {
config := state["config"].(config)
instance := state["instance"].(*ec2.Instance)
return fmt.Sprintf("%s:%d", instance.DNSName, config.SSHPort), nil
}
func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
config := state["config"].(config)
privateKey := state["privateKey"].(string)
keyring := new(ssh.SimpleKeychain)
if err := keyring.AddPEMKey(privateKey); err != nil {
return nil, fmt.Errorf("Error setting up SSH config: %s", err)
}
return &gossh.ClientConfig{
User: config.SSHUsername,
Auth: []gossh.ClientAuth{
gossh.ClientAuthKeyring(keyring),
},
}, nil
}
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"bytes"
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"cgl.tideland.biz/identifier"
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"fmt"
@@ -29,6 +29,19 @@ func (s *stepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
}
ui.Say("Launching a source AWS instance...")
imageResp, err := ec2conn.Images([]string{config.SourceAmi}, ec2.NewFilter())
if err != nil {
state["error"] = fmt.Errorf("There was a problem with the source AMI: %s", err)
return multistep.ActionHalt
}
if imageResp.Images[0].RootDeviceType != "ebs" {
state["error"] = fmt.Errorf(
"The provided source AMI is instance-store based. The\n" +
"amazon-ebs bundler can only work with EBS based AMIs.")
return multistep.ActionHalt
}
runResp, err := ec2conn.RunInstances(runOpts)
if err != nil {
err := fmt.Errorf("Error launching source instance: %s", err)
@@ -40,10 +53,10 @@ func (s *stepRunSourceInstance) Run(state map[string]interface{}) multistep.Step
s.instance = &runResp.Instances[0]
log.Printf("instance id: %s", s.instance.InstanceId)
ui.Say("Waiting for instance to become ready...")
ui.Say(fmt.Sprintf("Waiting for instance (%s) to become ready...", s.instance.InstanceId))
s.instance, err = waitForState(ec2conn, s.instance, []string{"pending"}, "running")
if err != nil {
err := fmt.Errorf("Error waiting for instance to become ready: %s", err)
err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", s.instance.InstanceId, err)
state["error"] = err
ui.Error(err.Error())
return multistep.ActionHalt
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"cgl.tideland.biz/identifier"
@@ -19,6 +19,12 @@ func (s *stepSecurityGroup) Run(state map[string]interface{}) multistep.StepActi
ec2conn := state["ec2"].(*ec2.EC2)
ui := state["ui"].(packer.Ui)
if config.SecurityGroupId != "" {
log.Printf("Using specified security group: %s", config.SecurityGroupId)
state["securityGroupId"] = config.SecurityGroupId
return multistep.ActionContinue
}
// Create the group
ui.Say("Creating temporary security group for this instance...")
groupName := fmt.Sprintf("packer %s", hex.EncodeToString(identifier.NewUUID().Raw()))
@@ -1,4 +1,4 @@
package amazonebs
package ebs
import (
"fmt"
-151
View File
@@ -1,151 +0,0 @@
package amazonebs
import (
gossh "code.google.com/p/go.crypto/ssh"
"errors"
"fmt"
"github.com/mitchellh/goamz/ec2"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/communicator/ssh"
"github.com/mitchellh/packer/packer"
"log"
"net"
"time"
)
type stepConnectSSH struct {
cancel bool
conn net.Conn
}
func (s *stepConnectSSH) Run(state map[string]interface{}) multistep.StepAction {
config := state["config"].(config)
ui := state["ui"].(packer.Ui)
var comm packer.Communicator
var err error
waitDone := make(chan bool, 1)
go func() {
comm, err = s.waitForSSH(state)
waitDone <- true
}()
log.Printf("Waiting for SSH, up to timeout: %s", config.SSHTimeout.String())
timeout := time.After(config.SSHTimeout)
WaitLoop:
for {
// Wait for either SSH to become available, a timeout to occur,
// or an interrupt to come through.
select {
case <-waitDone:
if err != nil {
ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
return multistep.ActionHalt
}
state["communicator"] = comm
break WaitLoop
case <-timeout:
ui.Error("Timeout waiting for SSH.")
s.cancel = true
return multistep.ActionHalt
case <-time.After(1 * time.Second):
if _, ok := state[multistep.StateCancelled]; ok {
log.Println("Interrupt detected, quitting waiting for SSH.")
return multistep.ActionHalt
}
}
}
return multistep.ActionContinue
}
func (s *stepConnectSSH) Cleanup(map[string]interface{}) {
if s.conn != nil {
s.conn.Close()
s.conn = nil
}
}
// This blocks until SSH becomes available, and sends the communicator
// on the given channel.
func (s *stepConnectSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
config := state["config"].(config)
instance := state["instance"].(*ec2.Instance)
privateKey := state["privateKey"].(string)
ui := state["ui"].(packer.Ui)
// Build the keyring for authentication. This stores the private key
// we'll use to authenticate.
keyring := &ssh.SimpleKeychain{}
err := keyring.AddPEMKey(privateKey)
if err != nil {
return nil, fmt.Errorf("Error setting up SSH config: %s", err)
}
ui.Say("Waiting for SSH to become available...")
var comm packer.Communicator
var nc net.Conn
for {
if nc != nil {
nc.Close()
}
time.Sleep(5 * time.Second)
if s.cancel {
log.Println("SSH wait cancelled. Exiting loop.")
return nil, errors.New("SSH wait cancelled")
}
// Attempt to connect to SSH port
log.Printf(
"Opening TCP conn for SSH to %s:%d",
instance.DNSName, config.SSHPort)
nc, err := net.Dial("tcp",
fmt.Sprintf("%s:%d", instance.DNSName, config.SSHPort))
if err != nil {
log.Printf("TCP connection to SSH ip/port failed: %s", err)
continue
}
// Build the actual SSH client configuration
sshConfig := &gossh.ClientConfig{
User: config.SSHUsername,
Auth: []gossh.ClientAuth{
gossh.ClientAuthKeyring(keyring),
},
}
sshConnectSuccess := make(chan bool, 1)
go func() {
comm, err = ssh.New(nc, sshConfig)
if err != nil {
log.Printf("SSH connection fail: %s", err)
sshConnectSuccess <- false
return
}
sshConnectSuccess <- true
}()
select {
case success := <-sshConnectSuccess:
if !success {
continue
}
case <-time.After(5 * time.Second):
log.Printf("SSH handshake timeout. Trying again.")
continue
}
ui.Say("Connected via SSH!")
break
}
// Store the connection so we can close it later
s.conn = nc
return comm, nil
}
-25
View File
@@ -1,25 +0,0 @@
package amazonebs
import (
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/packer"
"log"
)
type stepProvision struct{}
func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
comm := state["communicator"].(packer.Communicator)
hook := state["hook"].(packer.Hook)
ui := state["ui"].(packer.Ui)
log.Println("Running the provision hook")
if err := hook.Run(packer.HookProvision, ui, comm, nil); err != nil {
state["error"] = err
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func (*stepProvision) Cleanup(map[string]interface{}) {}
+18
View File
@@ -2,6 +2,9 @@ package common
import (
"bytes"
"crypto/md5"
"crypto/sha1"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
@@ -46,6 +49,21 @@ type DownloadClient struct {
downloader Downloader
}
// HashForType returns the Hash implementation for the given string
// type, or nil if the type is not supported.
func HashForType(t string) hash.Hash {
switch t {
case "md5":
return md5.New()
case "sha1":
return sha1.New()
case "sha256":
return sha256.New()
default:
return nil
}
}
// NewDownloadClient returns a new DownloadClient for the given
// configuration.
func NewDownloadClient(c *DownloadConfig) *DownloadClient {
+45
View File
@@ -40,3 +40,48 @@ func TestDownloadClient_VerifyChecksum(t *testing.T) {
t.Fatal("didn't verify")
}
}
func TestHashForType(t *testing.T) {
if h := HashForType("md5"); h == nil {
t.Fatalf("md5 hash is nil")
} else {
h.Write([]byte("foo"))
result := h.Sum(nil)
expected := "acbd18db4cc2f85cedef654fccc4a4d8"
actual := hex.EncodeToString(result)
if actual != expected {
t.Fatalf("bad hash: %s", actual)
}
}
if h := HashForType("sha1"); h == nil {
t.Fatalf("sha1 hash is nil")
} else {
h.Write([]byte("foo"))
result := h.Sum(nil)
expected := "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"
actual := hex.EncodeToString(result)
if actual != expected {
t.Fatalf("bad hash: %s", actual)
}
}
if h := HashForType("sha256"); h == nil {
t.Fatalf("sha256 hash is nil")
} else {
h.Write([]byte("foo"))
result := h.Sum(nil)
expected := "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"
actual := hex.EncodeToString(result)
if actual != expected {
t.Fatalf("bad hash: %s", actual)
}
}
if HashForType("fake") != nil {
t.Fatalf("fake hash is not nil")
}
}
+157
View File
@@ -0,0 +1,157 @@
package common
import (
gossh "code.google.com/p/go.crypto/ssh"
"errors"
"fmt"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/communicator/ssh"
"github.com/mitchellh/packer/packer"
"log"
"strings"
"time"
)
// StepConnectSSH is a multistep Step implementation that waits for SSH
// to become available. It gets the connection information from a single
// configuration when creating the step.
//
// Uses:
// ui packer.Ui
//
// Produces:
// communicator packer.Communicator
type StepConnectSSH struct {
// SSHAddress is a function that returns the TCP address to connect to
// for SSH. This is a function so that you can query information
// if necessary for this address.
SSHAddress func(map[string]interface{}) (string, error)
// SSHConfig is a function that returns the proper client configuration
// for SSH access.
SSHConfig func(map[string]interface{}) (*gossh.ClientConfig, error)
// SSHWaitTimeout is the total timeout to wait for SSH to become available.
SSHWaitTimeout time.Duration
cancel bool
comm packer.Communicator
}
func (s *StepConnectSSH) Run(state map[string]interface{}) multistep.StepAction {
ui := state["ui"].(packer.Ui)
var comm packer.Communicator
var err error
waitDone := make(chan bool, 1)
go func() {
ui.Say("Waiting for SSH to become available...")
comm, err = s.waitForSSH(state)
waitDone <- true
}()
log.Printf("Waiting for SSH, up to timeout: %s", s.SSHWaitTimeout)
timeout := time.After(s.SSHWaitTimeout)
WaitLoop:
for {
// Wait for either SSH to become available, a timeout to occur,
// or an interrupt to come through.
select {
case <-waitDone:
if err != nil {
ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
return multistep.ActionHalt
}
ui.Say("Connected to SSH!")
s.comm = comm
state["communicator"] = comm
break WaitLoop
case <-timeout:
ui.Error("Timeout waiting for SSH.")
s.cancel = true
return multistep.ActionHalt
case <-time.After(1 * time.Second):
if _, ok := state[multistep.StateCancelled]; ok {
// The step sequence was cancelled, so cancel waiting for SSH
// and just start the halting process.
s.cancel = true
log.Println("Interrupt detected, quitting waiting for SSH.")
return multistep.ActionHalt
}
}
}
return multistep.ActionContinue
}
func (s *StepConnectSSH) Cleanup(map[string]interface{}) {
}
func (s *StepConnectSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
handshakeAttempts := 0
var comm packer.Communicator
for {
time.Sleep(5 * time.Second)
if s.cancel {
log.Println("SSH wait cancelled. Exiting loop.")
return nil, errors.New("SSH wait cancelled")
}
// First we request the TCP connection information
address, err := s.SSHAddress(state)
if err != nil {
log.Printf("Error getting SSH address: %s", err)
continue
}
// Retrieve the SSH configuration
sshConfig, err := s.SSHConfig(state)
if err != nil {
log.Printf("Error getting SSH config: %s", err)
continue
}
// Attempt to connect to SSH port
connFunc := ssh.ConnectFunc("tcp", address, 5*time.Minute)
nc, err := connFunc()
if err != nil {
log.Printf("TCP connection to SSH ip/port failed: %s", err)
continue
}
nc.Close()
// Then we attempt to connect via SSH
config := &ssh.Config{
Connection: connFunc,
SSHConfig: sshConfig,
}
comm, err = ssh.New(config)
if err != nil {
log.Printf("SSH handshake err: %s", err)
// Only count this as an attempt if we were able to attempt
// to authenticate. Note this is very brittle since it depends
// on the string of the error... but I don't see any other way.
if strings.Contains(err.Error(), "authenticate") {
log.Printf("Detected authentication error. Increasing handshake attempts.")
handshakeAttempts += 1
}
if handshakeAttempts < 10 {
// Try to connect via SSH a handful of times
continue
}
return nil, err
}
break
}
return comm, nil
}
+14
View File
@@ -0,0 +1,14 @@
package common
import (
"github.com/mitchellh/multistep"
"testing"
)
func TestStepConnectSSH_Impl(t *testing.T) {
var raw interface{}
raw = new(StepConnectSSH)
if _, ok := raw.(multistep.Step); !ok {
t.Fatalf("connect ssh should be a step")
}
}
+135
View File
@@ -0,0 +1,135 @@
package common
import (
"fmt"
"github.com/mitchellh/go-fs"
"github.com/mitchellh/go-fs/fat"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/packer"
"io"
"io/ioutil"
"log"
"os"
"path/filepath"
)
// StepCreateFloppy will create a floppy disk with the given files.
// The floppy disk doesn't support sub-directories. Only files at the
// root level are supported.
type StepCreateFloppy struct {
Files []string
floppyPath string
}
func (s *StepCreateFloppy) Run(state map[string]interface{}) multistep.StepAction {
if len(s.Files) == 0 {
log.Println("No floppy files specified. Floppy disk will not be made.")
return multistep.ActionContinue
}
ui := state["ui"].(packer.Ui)
ui.Say("Creating floppy disk...")
// Create a temporary file to be our floppy drive
floppyF, err := ioutil.TempFile("", "packer")
if err != nil {
state["error"] = fmt.Errorf("Error creating temporary file for floppy: %s", err)
return multistep.ActionHalt
}
defer floppyF.Close()
// Set the path so we can remove it later
s.floppyPath = floppyF.Name()
log.Printf("Floppy path: %s", floppyF.Name())
// Set the size of the file to be a floppy sized
if err := floppyF.Truncate(1440 * 1024); err != nil {
state["error"] = fmt.Errorf("Error creating floppy: %s", err)
return multistep.ActionHalt
}
// BlockDevice backed by the file for our filesystem
log.Println("Initializing block device backed by temporary file")
device, err := fs.NewFileDisk(floppyF)
if err != nil {
state["error"] = fmt.Errorf("Error creating floppy: %s", err)
return multistep.ActionHalt
}
// Format the block device so it contains a valid FAT filesystem
log.Println("Formatting the block device with a FAT filesystem...")
formatConfig := &fat.SuperFloppyConfig{
FATType: fat.FAT12,
Label: "packer",
OEMName: "packer",
}
if fat.FormatSuperFloppy(device, formatConfig); err != nil {
state["error"] = fmt.Errorf("Error creating floppy: %s", err)
return multistep.ActionHalt
}
// The actual FAT filesystem
log.Println("Initializing FAT filesystem on block device")
fatFs, err := fat.New(device)
if err != nil {
state["error"] = fmt.Errorf("Error creating floppy: %s", err)
return multistep.ActionHalt
}
// Get the root directory to the filesystem
log.Println("Reading the root directory from the filesystem")
rootDir, err := fatFs.RootDir()
if err != nil {
state["error"] = fmt.Errorf("Error creating floppy: %s", err)
return multistep.ActionHalt
}
// Go over each file and copy it.
for _, filename := range s.Files {
ui.Message(fmt.Sprintf("Copying: %s", filepath.Base(filename)))
if s.addSingleFile(rootDir, filename); err != nil {
state["error"] = fmt.Errorf("Error adding file to floppy: %s", err)
return multistep.ActionHalt
}
}
// Set the path to the floppy so it can be used later
state["floppy_path"] = s.floppyPath
return multistep.ActionContinue
}
func (s *StepCreateFloppy) Cleanup(map[string]interface{}) {
if s.floppyPath != "" {
log.Printf("Deleting floppy disk: %s", s.floppyPath)
os.Remove(s.floppyPath)
}
}
func (s *StepCreateFloppy) addSingleFile(dir fs.Directory, src string) error {
log.Printf("Adding file to floppy: %s", src)
inputF, err := os.Open(src)
if err != nil {
return err
}
defer inputF.Close()
entry, err := dir.AddFile(filepath.Base(src))
if err != nil {
return err
}
fatFile, err := entry.File()
if err != nil {
return err
}
if _, err := io.Copy(fatFile, inputF); err != nil {
return err
}
return nil
}
@@ -1,4 +1,4 @@
package digitalocean
package common
import (
"github.com/mitchellh/multistep"
@@ -6,9 +6,18 @@ import (
"log"
)
type stepProvision struct{}
// StepProvision runs the provisioners.
//
// Uses:
// communicator packer.Communicator
// hook packer.Hook
// ui packer.Ui
//
// Produces:
// <nothing>
type StepProvision struct{}
func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
func (*StepProvision) Run(state map[string]interface{}) multistep.StepAction {
comm := state["communicator"].(packer.Communicator)
hook := state["hook"].(packer.Hook)
ui := state["ui"].(packer.Ui)
@@ -22,4 +31,4 @@ func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
return multistep.ActionContinue
}
func (*stepProvision) Cleanup(map[string]interface{}) {}
func (*StepProvision) Cleanup(map[string]interface{}) {}
+14
View File
@@ -0,0 +1,14 @@
package common
import (
"github.com/mitchellh/multistep"
"testing"
)
func TestStepProvision_Impl(t *testing.T) {
var raw interface{}
raw = new(StepProvision)
if _, ok := raw.(multistep.Step); !ok {
t.Fatalf("provision should be a step")
}
}
+54 -14
View File
@@ -12,7 +12,10 @@ import (
"github.com/mitchellh/packer/builder/common"
"github.com/mitchellh/packer/packer"
"log"
"os"
"sort"
"strconv"
"strings"
"text/template"
"time"
)
@@ -37,9 +40,6 @@ type config struct {
SnapshotName string
SSHUsername string `mapstructure:"ssh_username"`
SSHPort uint `mapstructure:"ssh_port"`
SSHTimeout time.Duration
EventDelay time.Duration
StateTimeout time.Duration
PackerDebug bool `mapstructure:"packer_debug"`
@@ -47,6 +47,12 @@ type config struct {
RawSSHTimeout string `mapstructure:"ssh_timeout"`
RawEventDelay string `mapstructure:"event_delay"`
RawStateTimeout string `mapstructure:"state_timeout"`
// These are unexported since they're set by other fields
// being set.
sshTimeout time.Duration
eventDelay time.Duration
stateTimeout time.Duration
}
type Builder struct {
@@ -55,15 +61,49 @@ type Builder struct {
}
func (b *Builder) Prepare(raws ...interface{}) error {
var md mapstructure.Metadata
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &b.config,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return err
}
for _, raw := range raws {
err := mapstructure.Decode(raw, &b.config)
err := decoder.Decode(raw)
if err != nil {
return err
}
}
// Accumulate any errors
errs := make([]error, 0)
// Unused keys are errors
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
if unused != "type" && !strings.HasPrefix(unused, "packer_") {
errs = append(
errs, fmt.Errorf("Unknown configuration key: %s", unused))
}
}
}
// Optional configuration with defaults
//
if b.config.APIKey == "" {
// Default to environment variable for api_key, if it exists
b.config.APIKey = os.Getenv("DIGITALOCEAN_API_KEY")
}
if b.config.ClientID == "" {
// Default to environment variable for client_id, if it exists
b.config.ClientID = os.Getenv("DIGITALOCEAN_CLIENT_ID")
}
if b.config.RegionID == 0 {
// Default to Region "New York"
b.config.RegionID = 1
@@ -112,11 +152,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.RawStateTimeout = "6m"
}
// A list of errors on the configuration
errs := make([]error, 0)
// Required configurations that will display errors if not set
//
if b.config.ClientID == "" {
errs = append(errs, errors.New("a client_id must be specified"))
}
@@ -129,19 +165,19 @@ func (b *Builder) Prepare(raws ...interface{}) error {
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing ssh_timeout: %s", err))
}
b.config.SSHTimeout = sshTimeout
b.config.sshTimeout = sshTimeout
eventDelay, err := time.ParseDuration(b.config.RawEventDelay)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing event_delay: %s", err))
}
b.config.EventDelay = eventDelay
b.config.eventDelay = eventDelay
stateTimeout, err := time.ParseDuration(b.config.RawStateTimeout)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing state_timeout: %s", err))
}
b.config.StateTimeout = stateTimeout
b.config.stateTimeout = stateTimeout
// Parse the name of the snapshot
snapNameBuf := new(bytes.Buffer)
@@ -180,8 +216,12 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
new(stepCreateSSHKey),
new(stepCreateDroplet),
new(stepDropletInfo),
new(stepConnectSSH),
new(stepProvision),
&common.StepConnectSSH{
SSHAddress: sshAddress,
SSHConfig: sshConfig,
SSHWaitTimeout: 5 * time.Minute,
},
new(common.StepProvision),
new(stepPowerOff),
new(stepSnapshot),
}
+37
View File
@@ -2,10 +2,17 @@ package digitalocean
import (
"github.com/mitchellh/packer/packer"
"os"
"strconv"
"testing"
)
func init() {
// Clear out the credential env vars
os.Setenv("DIGITALOCEAN_API_KEY", "")
os.Setenv("DIGITALOCEAN_CLIENT_ID", "")
}
func testConfig() map[string]interface{} {
return map[string]interface{}{
"client_id": "foo",
@@ -55,6 +62,15 @@ func TestBuilderPrepare_APIKey(t *testing.T) {
if err == nil {
t.Fatal("should have error")
}
// Test env variable
delete(config, "api_key")
os.Setenv("DIGITALOCEAN_API_KEY", "foo")
defer os.Setenv("DIGITALOCEAN_API_KEY", "")
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
}
func TestBuilderPrepare_ClientID(t *testing.T) {
@@ -79,6 +95,27 @@ func TestBuilderPrepare_ClientID(t *testing.T) {
if err == nil {
t.Fatal("should have error")
}
// Test env variable
delete(config, "client_id")
os.Setenv("DIGITALOCEAN_CLIENT_ID", "foo")
defer os.Setenv("DIGITALOCEAN_CLIENT_ID", "")
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
}
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_RegionID(t *testing.T) {
+30
View File
@@ -0,0 +1,30 @@
package digitalocean
import (
gossh "code.google.com/p/go.crypto/ssh"
"fmt"
"github.com/mitchellh/packer/communicator/ssh"
)
func sshAddress(state map[string]interface{}) (string, error) {
config := state["config"].(config)
ipAddress := state["droplet_ip"].(string)
return fmt.Sprintf("%s:%d", ipAddress, config.SSHPort), nil
}
func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
config := state["config"].(config)
privateKey := state["privateKey"].(string)
keyring := new(ssh.SimpleKeychain)
if err := keyring.AddPEMKey(privateKey); err != nil {
return nil, fmt.Errorf("Error setting up SSH config: %s", err)
}
return &gossh.ClientConfig{
User: config.SSHUsername,
Auth: []gossh.ClientAuth{
gossh.ClientAuthKeyring(keyring),
},
}, nil
}
-137
View File
@@ -1,137 +0,0 @@
package digitalocean
import (
gossh "code.google.com/p/go.crypto/ssh"
"errors"
"fmt"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/communicator/ssh"
"github.com/mitchellh/packer/packer"
"log"
"net"
"time"
)
type stepConnectSSH struct {
conn net.Conn
}
func (s *stepConnectSSH) Run(state map[string]interface{}) multistep.StepAction {
config := state["config"].(config)
privateKey := state["privateKey"].(string)
ui := state["ui"].(packer.Ui)
ipAddress := state["droplet_ip"]
// Build the keyring for authentication. This stores the private key
// we'll use to authenticate.
keyring := &ssh.SimpleKeychain{}
err := keyring.AddPEMKey(privateKey)
if err != nil {
err := fmt.Errorf("Error setting up SSH config: %s", err)
state["error"] = err
ui.Error(err.Error())
return multistep.ActionHalt
}
// Build the actual SSH client configuration
sshConfig := &gossh.ClientConfig{
User: config.SSHUsername,
Auth: []gossh.ClientAuth{
gossh.ClientAuthKeyring(keyring),
},
}
// Start trying to connect to SSH
connected := make(chan error, 1)
connectQuit := make(chan bool, 1)
defer func() {
connectQuit <- true
}()
var comm packer.Communicator
go func() {
var err error
ui.Say("Connecting to the droplet via SSH...")
attempts := 0
handshakeAttempts := 0
for {
select {
case <-connectQuit:
return
default:
}
attempts += 1
log.Printf(
"Opening TCP conn for SSH to %s:%d (attempt %d)",
ipAddress, config.SSHPort, attempts)
s.conn, err = net.DialTimeout(
"tcp",
fmt.Sprintf("%s:%d", ipAddress, config.SSHPort),
10*time.Second)
if err == nil {
log.Println("TCP connection made. Attempting SSH handshake.")
comm, err = ssh.New(s.conn, sshConfig)
if err == nil {
log.Println("Connected to SSH!")
break
}
handshakeAttempts += 1
log.Printf("SSH handshake error: %s", err)
if handshakeAttempts > 5 {
connected <- err
return
}
}
// A brief sleep so we're not being overly zealous attempting
// to connect to the instance.
time.Sleep(500 * time.Millisecond)
}
connected <- nil
}()
log.Printf("Waiting up to %s for SSH connection", config.SSHTimeout)
timeout := time.After(config.SSHTimeout)
ConnectWaitLoop:
for {
select {
case err := <-connected:
if err != nil {
err := fmt.Errorf("Error connecting to SSH: %s", err)
state["error"] = err
ui.Error(err.Error())
return multistep.ActionHalt
}
// We connected. Just break the loop.
break ConnectWaitLoop
case <-timeout:
err := errors.New("Timeout waiting for SSH to become available.")
state["error"] = err
ui.Error(err.Error())
return multistep.ActionHalt
case <-time.After(1 * time.Second):
if _, ok := state[multistep.StateCancelled]; ok {
log.Println("Interrupt detected, quitting waiting for SSH.")
return multistep.ActionHalt
}
}
}
// Set the communicator on the state bag so it can be used later
state["communicator"] = comm
return multistep.ActionContinue
}
func (s *stepConnectSSH) Cleanup(map[string]interface{}) {
if s.conn != nil {
s.conn.Close()
}
}
+1 -1
View File
@@ -60,7 +60,7 @@ func (s *stepCreateDroplet) Cleanup(state map[string]interface{}) {
// Otherwise we get "pending event" errors, even though there isn't
// one.
log.Printf("Sleeping for %v, event_delay", c.RawEventDelay)
time.Sleep(c.EventDelay)
time.Sleep(c.eventDelay)
err := client.DestroyDroplet(s.dropletId)
+1 -1
View File
@@ -20,7 +20,7 @@ func (s *stepPowerOff) Run(state map[string]interface{}) multistep.StepAction {
// Otherwise we get "pending event" errors, even though there isn't
// one.
log.Printf("Sleeping for %v, event_delay", c.RawEventDelay)
time.Sleep(c.EventDelay)
time.Sleep(c.eventDelay)
// Poweroff the droplet so it can be snapshot
err := client.PowerOffDroplet(dropletId)
+1 -3
View File
@@ -17,9 +17,7 @@ func waitForDropletState(desiredState string, dropletId uint, client *DigitalOce
attempts += 1
log.Printf("Checking droplet status... (attempt: %d)", attempts)
_, status, err := client.DropletStatus(dropletId)
if err != nil {
log.Println(err)
break
@@ -37,7 +35,7 @@ func waitForDropletState(desiredState string, dropletId uint, client *DigitalOce
}()
log.Printf("Waiting for up to %s for droplet to become %s", c.RawStateTimeout, desiredState)
timeout := time.After(c.StateTimeout)
timeout := time.After(c.stateTimeout)
ActiveWaitLoop:
for {
+92 -39
View File
@@ -12,6 +12,7 @@ import (
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
"time"
)
@@ -25,54 +26,85 @@ type Builder struct {
}
type config struct {
BootCommand []string `mapstructure:"boot_command"`
BootWait time.Duration ``
DiskSize uint `mapstructure:"disk_size"`
GuestAdditionsPath string `mapstructure:"guest_additions_path"`
GuestAdditionsURL string `mapstructure:"guest_additions_url"`
GuestAdditionsSHA256 string `mapstructure:"guest_additions_sha256"`
GuestOSType string `mapstructure:"guest_os_type"`
Headless bool `mapstructure:"headless"`
HTTPDir string `mapstructure:"http_directory"`
HTTPPortMin uint `mapstructure:"http_port_min"`
HTTPPortMax uint `mapstructure:"http_port_max"`
ISOMD5 string `mapstructure:"iso_md5"`
ISOUrl string `mapstructure:"iso_url"`
OutputDir string `mapstructure:"output_directory"`
ShutdownCommand string `mapstructure:"shutdown_command"`
ShutdownTimeout time.Duration ``
SSHHostPortMin uint `mapstructure:"ssh_host_port_min"`
SSHHostPortMax uint `mapstructure:"ssh_host_port_max"`
SSHPassword string `mapstructure:"ssh_password"`
SSHPort uint `mapstructure:"ssh_port"`
SSHUser string `mapstructure:"ssh_username"`
SSHWaitTimeout time.Duration ``
VBoxVersionFile string `mapstructure:"virtualbox_version_file"`
VBoxManage [][]string `mapstructure:"vboxmanage"`
VMName string `mapstructure:"vm_name"`
BootCommand []string `mapstructure:"boot_command"`
DiskSize uint `mapstructure:"disk_size"`
FloppyFiles []string `mapstructure:"floppy_files"`
GuestAdditionsPath string `mapstructure:"guest_additions_path"`
GuestAdditionsURL string `mapstructure:"guest_additions_url"`
GuestAdditionsSHA256 string `mapstructure:"guest_additions_sha256"`
GuestOSType string `mapstructure:"guest_os_type"`
Headless bool `mapstructure:"headless"`
HTTPDir string `mapstructure:"http_directory"`
HTTPPortMin uint `mapstructure:"http_port_min"`
HTTPPortMax uint `mapstructure:"http_port_max"`
ISOChecksum string `mapstructure:"iso_checksum"`
ISOChecksumType string `mapstructure:"iso_checksum_type"`
ISOUrl string `mapstructure:"iso_url"`
OutputDir string `mapstructure:"output_directory"`
ShutdownCommand string `mapstructure:"shutdown_command"`
SSHHostPortMin uint `mapstructure:"ssh_host_port_min"`
SSHHostPortMax uint `mapstructure:"ssh_host_port_max"`
SSHPassword string `mapstructure:"ssh_password"`
SSHPort uint `mapstructure:"ssh_port"`
SSHUser string `mapstructure:"ssh_username"`
VBoxVersionFile string `mapstructure:"virtualbox_version_file"`
VBoxManage [][]string `mapstructure:"vboxmanage"`
VMName string `mapstructure:"vm_name"`
PackerBuildName string `mapstructure:"packer_build_name"`
PackerDebug bool `mapstructure:"packer_debug"`
PackerForce bool `mapstructure:"packer_force"`
RawBootWait string `mapstructure:"boot_wait"`
RawShutdownTimeout string `mapstructure:"shutdown_timeout"`
RawSSHWaitTimeout string `mapstructure:"ssh_wait_timeout"`
bootWait time.Duration ``
shutdownTimeout time.Duration ``
sshWaitTimeout time.Duration ``
}
func (b *Builder) Prepare(raws ...interface{}) error {
var err error
var md mapstructure.Metadata
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &b.config,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return err
}
for _, raw := range raws {
err := mapstructure.Decode(raw, &b.config)
err := decoder.Decode(raw)
if err != nil {
return err
}
}
// Accumulate any errors
errs := make([]error, 0)
// Unused keys are errors
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
if unused != "type" && !strings.HasPrefix(unused, "packer_") {
errs = append(
errs, fmt.Errorf("Unknown configuration key: %s", unused))
}
}
}
if b.config.DiskSize == 0 {
b.config.DiskSize = 40000
}
if b.config.FloppyFiles == nil {
b.config.FloppyFiles = make([]string, 0)
}
if b.config.GuestAdditionsPath == "" {
b.config.GuestAdditionsPath = "VBoxGuestAdditions.iso"
}
@@ -121,16 +153,25 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.VMName = fmt.Sprintf("packer-%s", b.config.PackerBuildName)
}
errs := make([]error, 0)
if b.config.HTTPPortMin > b.config.HTTPPortMax {
errs = append(errs, errors.New("http_port_min must be less than http_port_max"))
}
if b.config.ISOMD5 == "" {
errs = append(errs, errors.New("Due to large file sizes, an iso_md5 is required"))
if b.config.ISOChecksum == "" {
errs = append(errs, errors.New("Due to large file sizes, an iso_checksum is required"))
} else {
b.config.ISOMD5 = strings.ToLower(b.config.ISOMD5)
b.config.ISOChecksum = strings.ToLower(b.config.ISOChecksum)
}
if b.config.ISOChecksumType == "" {
errs = append(errs, errors.New("The iso_checksum_type must be specified."))
} else {
b.config.ISOChecksumType = strings.ToLower(b.config.ISOChecksumType)
if h := common.HashForType(b.config.ISOChecksumType); h == nil {
errs = append(
errs,
fmt.Errorf("Unsupported checksum type: %s", b.config.ISOChecksumType))
}
}
if b.config.ISOUrl == "" {
@@ -213,11 +254,15 @@ func (b *Builder) Prepare(raws ...interface{}) error {
}
}
if _, err := os.Stat(b.config.OutputDir); err == nil {
errs = append(errs, errors.New("Output directory already exists. It must not exist."))
if !b.config.PackerForce {
if _, err := os.Stat(b.config.OutputDir); err == nil {
errs = append(
errs,
errors.New("Output directory already exists. It must not exist."))
}
}
b.config.BootWait, err = time.ParseDuration(b.config.RawBootWait)
b.config.bootWait, err = time.ParseDuration(b.config.RawBootWait)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing boot_wait: %s", err))
}
@@ -230,7 +275,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.RawSSHWaitTimeout = "20m"
}
b.config.ShutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
b.config.shutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing shutdown_timeout: %s", err))
}
@@ -243,7 +288,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
errs = append(errs, errors.New("An ssh_username must be specified."))
}
b.config.SSHWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
b.config.sshWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing ssh_wait_timeout: %s", err))
}
@@ -265,19 +310,27 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
new(stepDownloadGuestAdditions),
new(stepDownloadISO),
new(stepPrepareOutputDir),
&common.StepCreateFloppy{
Files: b.config.FloppyFiles,
},
new(stepHTTPServer),
new(stepSuppressMessages),
new(stepCreateVM),
new(stepCreateDisk),
new(stepAttachISO),
new(stepAttachFloppy),
new(stepForwardSSH),
new(stepVBoxManage),
new(stepRun),
new(stepTypeBootCommand),
new(stepWaitForSSH),
&common.StepConnectSSH{
SSHAddress: sshAddress,
SSHConfig: sshConfig,
SSHWaitTimeout: b.config.sshWaitTimeout,
},
new(stepUploadVersion),
new(stepUploadGuestAdditions),
new(stepProvision),
new(common.StepProvision),
new(stepShutdown),
new(stepExport),
}
+78 -8
View File
@@ -10,9 +10,10 @@ import (
func testConfig() map[string]interface{} {
return map[string]interface{}{
"iso_md5": "foo",
"iso_url": "http://www.google.com/",
"ssh_username": "foo",
"iso_checksum": "foo",
"iso_checksum_type": "md5",
"iso_url": "http://www.google.com/",
"ssh_username": "foo",
packer.BuildNameConfigKey: "foo",
}
@@ -115,6 +116,33 @@ func TestBuilderPrepare_DiskSize(t *testing.T) {
}
}
func TestBuilderPrepare_FloppyFiles(t *testing.T) {
var b Builder
config := testConfig()
delete(config, "floppy_files")
err := b.Prepare(config)
if err != nil {
t.Fatalf("bad err: %s", err)
}
if len(b.config.FloppyFiles) != 0 {
t.Fatalf("bad: %#v", b.config.FloppyFiles)
}
config["floppy_files"] = []string{"foo", "bar"}
b = Builder{}
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
expected := []string{"foo", "bar"}
if !reflect.DeepEqual(b.config.FloppyFiles, expected) {
t.Fatalf("bad: %#v", b.config.FloppyFiles)
}
}
func TestBuilderPrepare_GuestAdditionsPath(t *testing.T) {
var b Builder
config := testConfig()
@@ -244,26 +272,68 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
}
}
func TestBuilderPrepare_ISOMD5(t *testing.T) {
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_ISOChecksum(t *testing.T) {
var b Builder
config := testConfig()
// Test bad
config["iso_md5"] = ""
config["iso_checksum"] = ""
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test good
config["iso_md5"] = "FOo"
config["iso_checksum"] = "FOo"
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.ISOMD5 != "foo" {
t.Fatalf("should've lowercased: %s", b.config.ISOMD5)
if b.config.ISOChecksum != "foo" {
t.Fatalf("should've lowercased: %s", b.config.ISOChecksum)
}
}
func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
var b Builder
config := testConfig()
// Test bad
config["iso_checksum_type"] = ""
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test good
config["iso_checksum_type"] = "mD5"
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.ISOChecksumType != "md5" {
t.Fatalf("should've lowercased: %s", b.config.ISOChecksumType)
}
// Test unknown
config["iso_checksum_type"] = "fake"
err = b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
+6
View File
@@ -59,6 +59,12 @@ func (d *VBox42Driver) IsRunning(name string) (bool, error) {
if line == `VMState="stopping"` {
return true, nil
}
// We consider "paused" to still be running. We wait for it to
// be completely stopped or some other state.
if line == `VMState="paused"` {
return true, nil
}
}
return false, nil
+25
View File
@@ -0,0 +1,25 @@
package virtualbox
import (
gossh "code.google.com/p/go.crypto/ssh"
"fmt"
"github.com/mitchellh/packer/communicator/ssh"
)
func sshAddress(state map[string]interface{}) (string, error) {
sshHostPort := state["sshHostPort"].(uint)
return fmt.Sprintf("127.0.0.1:%d", sshHostPort), nil
}
func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
config := state["config"].(*config)
return &gossh.ClientConfig{
User: config.SSHUser,
Auth: []gossh.ClientAuth{
gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
gossh.ClientAuthKeyboardInteractive(
ssh.PasswordKeyboardInteractive(config.SSHPassword)),
},
}, nil
}
+128
View File
@@ -0,0 +1,128 @@
package virtualbox
import (
"fmt"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/packer"
"io"
"io/ioutil"
"log"
"os"
"path/filepath"
)
// This step attaches the ISO to the virtual machine.
//
// Uses:
//
// Produces:
type stepAttachFloppy struct {
floppyPath string
}
func (s *stepAttachFloppy) Run(state map[string]interface{}) multistep.StepAction {
// Determine if we even have a floppy disk to attach
var floppyPath string
if floppyPathRaw, ok := state["floppy_path"]; ok {
floppyPath = floppyPathRaw.(string)
} else {
log.Println("No floppy disk, not attaching.")
return multistep.ActionContinue
}
// VirtualBox is really dumb and can't figure out the format of the file
// without an extension, so we need to add the "vfd" extension to the
// floppy.
floppyPath, err := s.copyFloppy(floppyPath)
if err != nil {
state["error"] = fmt.Errorf("Error preparing floppy: %s", err)
return multistep.ActionHalt
}
driver := state["driver"].(Driver)
ui := state["ui"].(packer.Ui)
vmName := state["vmName"].(string)
ui.Say("Attaching floppy disk...")
// Create the floppy disk controller
command := []string{
"storagectl", vmName,
"--name", "Floppy Controller",
"--add", "floppy",
}
if err := driver.VBoxManage(command...); err != nil {
state["error"] = fmt.Errorf("Error creating floppy controller: %s", err)
return multistep.ActionHalt
}
// Attach the floppy to the controller
command = []string{
"storageattach", vmName,
"--storagectl", "Floppy Controller",
"--port", "0",
"--device", "0",
"--type", "fdd",
"--medium", floppyPath,
}
if err := driver.VBoxManage(command...); err != nil {
state["error"] = fmt.Errorf("Error attaching floppy: %s", err)
return multistep.ActionHalt
}
// Track the path so that we can unregister it from VirtualBox later
s.floppyPath = floppyPath
return multistep.ActionContinue
}
func (s *stepAttachFloppy) Cleanup(state map[string]interface{}) {
if s.floppyPath == "" {
return
}
// Delete the floppy disk
defer os.Remove(s.floppyPath)
driver := state["driver"].(Driver)
vmName := state["vmName"].(string)
command := []string{
"storageattach", vmName,
"--storagectl", "Floppy Controller",
"--port", "0",
"--device", "0",
"--medium", "none",
}
if err := driver.VBoxManage(command...); err != nil {
log.Printf("Error unregistering floppy: %s", err)
}
}
func (s *stepAttachFloppy) copyFloppy(path string) (string, error) {
tempdir, err := ioutil.TempDir("", "packer")
if err != nil {
return "", err
}
floppyPath := filepath.Join(tempdir, "floppy.vfd")
f, err := os.Create(floppyPath)
if err != nil {
return "", err
}
defer f.Close()
sourceF, err := os.Open(path)
if err != nil {
return "", err
}
defer sourceF.Close()
log.Printf("Copying floppy to temp location: %s", floppyPath)
if _, err := io.Copy(f, sourceF); err != nil {
return "", err
}
return floppyPath, nil
}
+2 -3
View File
@@ -1,7 +1,6 @@
package virtualbox
import (
"crypto/md5"
"encoding/hex"
"fmt"
"github.com/mitchellh/multistep"
@@ -33,7 +32,7 @@ func (s *stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
checksum, err := hex.DecodeString(config.ISOMD5)
checksum, err := hex.DecodeString(config.ISOChecksum)
if err != nil {
state["error"] = fmt.Errorf("Error parsing checksum: %s", err)
return multistep.ActionHalt
@@ -47,7 +46,7 @@ func (s *stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
Url: config.ISOUrl,
TargetPath: cachePath,
CopyFile: false,
Hash: md5.New(),
Hash: common.HashForType(config.ISOChecksumType),
Checksum: checksum,
}
+19 -1
View File
@@ -22,7 +22,8 @@ func (s *stepExport) Run(state map[string]interface{}) multistep.StepAction {
vmName := state["vmName"].(string)
// Clear out the Packer-created forwarding rule
ui.Say(fmt.Sprintf("Deleting forwarded port mapping for SSH (host port %d)", state["sshHostPort"]))
ui.Say("Preparing to export machine...")
ui.Message(fmt.Sprintf("Deleting forwarded port mapping for SSH (host port %d)", state["sshHostPort"]))
command := []string{"modifyvm", vmName, "--natpf1", "delete", "packerssh"}
if err := driver.VBoxManage(command...); err != nil {
err := fmt.Errorf("Error deleting port forwarding rule: %s", err)
@@ -31,6 +32,23 @@ func (s *stepExport) Run(state map[string]interface{}) multistep.StepAction {
return multistep.ActionHalt
}
// Remove the attached floppy disk, if it exists
if _, ok := state["floppy_path"]; ok {
ui.Message("Removing floppy drive...")
command := []string{
"storageattach", vmName,
"--storagectl", "Floppy Controller",
"--port", "0",
"--device", "0",
"--medium", "none",
}
if err := driver.VBoxManage(command...); err != nil {
state["error"] = fmt.Errorf("Error removing floppy: %s", err)
return multistep.ActionHalt
}
}
// Export the VM to an OVF
outputPath := filepath.Join(config.OutputDir, "packer.ovf")
@@ -10,6 +10,12 @@ type stepPrepareOutputDir struct{}
func (stepPrepareOutputDir) Run(state map[string]interface{}) multistep.StepAction {
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
if _, err := os.Stat(config.OutputDir); err == nil && config.PackerForce {
ui.Say("Deleting previous output directory...")
os.RemoveAll(config.OutputDir)
}
if err := os.MkdirAll(config.OutputDir, 0755); err != nil {
state["error"] = err
-25
View File
@@ -1,25 +0,0 @@
package virtualbox
import (
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/packer"
"log"
)
type stepProvision struct{}
func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
comm := state["communicator"].(packer.Communicator)
hook := state["hook"].(packer.Hook)
ui := state["ui"].(packer.Ui)
log.Println("Running the provision hook")
if err := hook.Run(packer.HookProvision, ui, comm, nil); err != nil {
state["error"] = err
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func (*stepProvision) Cleanup(map[string]interface{}) {}
+3 -3
View File
@@ -40,9 +40,9 @@ func (s *stepRun) Run(state map[string]interface{}) multistep.StepAction {
s.vmName = vmName
if int64(config.BootWait) > 0 {
ui.Say(fmt.Sprintf("Waiting %s for boot...", config.BootWait))
time.Sleep(config.BootWait)
if int64(config.bootWait) > 0 {
ui.Say(fmt.Sprintf("Waiting %s for boot...", config.bootWait))
time.Sleep(config.bootWait)
}
return multistep.ActionContinue
+2 -2
View File
@@ -45,8 +45,8 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
cmd.Wait()
// Wait for the machine to actually shut down
log.Printf("Waiting max %s for shutdown to complete", config.ShutdownTimeout)
shutdownTimer := time.After(config.ShutdownTimeout)
log.Printf("Waiting max %s for shutdown to complete", config.shutdownTimeout)
shutdownTimer := time.After(config.shutdownTimeout)
for {
running, _ := driver.IsRunning(vmName)
if !running {
-149
View File
@@ -1,149 +0,0 @@
package virtualbox
import (
gossh "code.google.com/p/go.crypto/ssh"
"errors"
"fmt"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/communicator/ssh"
"github.com/mitchellh/packer/packer"
"log"
"net"
"time"
)
// This step waits for SSH to become available and establishes an SSH
// connection.
//
// Uses:
// config *config
// sshHostPort uint
// ui packer.Ui
//
// Produces:
// communicator packer.Communicator
type stepWaitForSSH struct {
cancel bool
conn net.Conn
}
func (s *stepWaitForSSH) Run(state map[string]interface{}) multistep.StepAction {
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
var comm packer.Communicator
var err error
waitDone := make(chan bool, 1)
go func() {
comm, err = s.waitForSSH(state)
waitDone <- true
}()
log.Printf("Waiting for SSH, up to timeout: %s", config.SSHWaitTimeout.String())
timeout := time.After(config.SSHWaitTimeout)
WaitLoop:
for {
// Wait for either SSH to become available, a timeout to occur,
// or an interrupt to come through.
select {
case <-waitDone:
if err != nil {
ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
return multistep.ActionHalt
}
state["communicator"] = comm
break WaitLoop
case <-timeout:
ui.Error("Timeout waiting for SSH.")
s.cancel = true
return multistep.ActionHalt
case <-time.After(1 * time.Second):
if _, ok := state[multistep.StateCancelled]; ok {
log.Println("Interrupt detected, quitting waiting for SSH.")
return multistep.ActionHalt
}
}
}
return multistep.ActionContinue
}
func (s *stepWaitForSSH) Cleanup(map[string]interface{}) {
if s.conn != nil {
s.conn.Close()
s.conn = nil
}
}
// This blocks until SSH becomes available, and sends the communicator
// on the given channel.
func (s *stepWaitForSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
sshHostPort := state["sshHostPort"].(uint)
ui.Say("Waiting for SSH to become available...")
var comm packer.Communicator
var nc net.Conn
for {
if nc != nil {
nc.Close()
}
time.Sleep(5 * time.Second)
if s.cancel {
log.Println("SSH wait cancelled. Exiting loop.")
return nil, errors.New("SSH wait cancelled")
}
// Attempt to connect to SSH port
nc, err := net.Dial("tcp", fmt.Sprintf("127.0.0.1:%d", sshHostPort))
if err != nil {
log.Printf("TCP connection to SSH ip/port failed: %s", err)
continue
}
// Then we attempt to connect via SSH
sshConfig := &gossh.ClientConfig{
User: config.SSHUser,
Auth: []gossh.ClientAuth{
gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
gossh.ClientAuthKeyboardInteractive(
ssh.PasswordKeyboardInteractive(config.SSHPassword)),
},
}
sshConnectSuccess := make(chan bool, 1)
go func() {
comm, err = ssh.New(nc, sshConfig)
if err != nil {
log.Printf("SSH connection fail: %s", err)
sshConnectSuccess <- false
return
}
sshConnectSuccess <- true
}()
select {
case success := <-sshConnectSuccess:
if !success {
continue
}
case <-time.After(5 * time.Second):
log.Printf("SSH handshake timeout. Trying again.")
continue
}
ui.Say("Connected via SSH!")
break
}
// Store the connection so we can close it later
s.conn = nc
return comm, nil
}
+72 -19
View File
@@ -12,6 +12,7 @@ import (
"net/url"
"os"
"path/filepath"
"sort"
"strings"
"text/template"
"time"
@@ -28,8 +29,10 @@ type Builder struct {
type config struct {
DiskName string `mapstructure:"vmdk_name"`
DiskSize uint `mapstructure:"disk_size"`
FloppyFiles []string `mapstructure:"floppy_files"`
GuestOSType string `mapstructure:"guest_os_type"`
ISOMD5 string `mapstructure:"iso_md5"`
ISOChecksum string `mapstructure:"iso_checksum"`
ISOChecksumType string `mapstructure:"iso_checksum_type"`
ISOUrl string `mapstructure:"iso_url"`
VMName string `mapstructure:"vm_name"`
OutputDir string `mapstructure:"output_directory"`
@@ -38,14 +41,11 @@ type config struct {
HTTPPortMin uint `mapstructure:"http_port_min"`
HTTPPortMax uint `mapstructure:"http_port_max"`
BootCommand []string `mapstructure:"boot_command"`
BootWait time.Duration ``
SkipCompaction bool `mapstructure:"skip_compaction"`
ShutdownCommand string `mapstructure:"shutdown_command"`
ShutdownTimeout time.Duration ``
SSHUser string `mapstructure:"ssh_username"`
SSHPassword string `mapstructure:"ssh_password"`
SSHPort uint `mapstructure:"ssh_port"`
SSHWaitTimeout time.Duration ``
ToolsUploadFlavor string `mapstructure:"tools_upload_flavor"`
ToolsUploadPath string `mapstructure:"tools_upload_path"`
VMXData map[string]string `mapstructure:"vmx_data"`
@@ -54,20 +54,50 @@ type config struct {
PackerBuildName string `mapstructure:"packer_build_name"`
PackerDebug bool `mapstructure:"packer_debug"`
PackerForce bool `mapstructure:"packer_force"`
RawBootWait string `mapstructure:"boot_wait"`
RawShutdownTimeout string `mapstructure:"shutdown_timeout"`
RawSSHWaitTimeout string `mapstructure:"ssh_wait_timeout"`
bootWait time.Duration ``
shutdownTimeout time.Duration ``
sshWaitTimeout time.Duration ``
}
func (b *Builder) Prepare(raws ...interface{}) error {
var md mapstructure.Metadata
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &b.config,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return err
}
for _, raw := range raws {
err := mapstructure.Decode(raw, &b.config)
err := decoder.Decode(raw)
if err != nil {
return err
}
}
// Accumulate any errors
errs := make([]error, 0)
// Unused keys are errors
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
if unused != "type" && !strings.HasPrefix(unused, "packer_") {
errs = append(
errs, fmt.Errorf("Unknown configuration key: %s", unused))
}
}
}
if b.config.DiskName == "" {
b.config.DiskName = "disk"
}
@@ -76,6 +106,10 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.DiskSize = 40000
}
if b.config.FloppyFiles == nil {
b.config.FloppyFiles = make([]string, 0)
}
if b.config.GuestOSType == "" {
b.config.GuestOSType = "other"
}
@@ -116,18 +150,25 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.ToolsUploadPath = "{{ .Flavor }}.iso"
}
// Accumulate any errors
var err error
errs := make([]error, 0)
if b.config.HTTPPortMin > b.config.HTTPPortMax {
errs = append(errs, errors.New("http_port_min must be less than http_port_max"))
}
if b.config.ISOMD5 == "" {
errs = append(errs, errors.New("Due to large file sizes, an iso_md5 is required"))
if b.config.ISOChecksum == "" {
errs = append(errs, errors.New("Due to large file sizes, an iso_checksum is required"))
} else {
b.config.ISOMD5 = strings.ToLower(b.config.ISOMD5)
b.config.ISOChecksum = strings.ToLower(b.config.ISOChecksum)
}
if b.config.ISOChecksumType == "" {
errs = append(errs, errors.New("The iso_checksum_type must be specified."))
} else {
b.config.ISOChecksumType = strings.ToLower(b.config.ISOChecksumType)
if h := common.HashForType(b.config.ISOChecksumType); h == nil {
errs = append(
errs,
fmt.Errorf("Unsupported checksum type: %s", b.config.ISOChecksumType))
}
}
if b.config.ISOUrl == "" {
@@ -169,8 +210,12 @@ func (b *Builder) Prepare(raws ...interface{}) error {
}
}
if _, err := os.Stat(b.config.OutputDir); err == nil {
errs = append(errs, errors.New("Output directory already exists. It must not exist."))
if !b.config.PackerForce {
if _, err := os.Stat(b.config.OutputDir); err == nil {
errs = append(
errs,
errors.New("Output directory already exists. It must not exist."))
}
}
if b.config.SSHUser == "" {
@@ -178,7 +223,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
}
if b.config.RawBootWait != "" {
b.config.BootWait, err = time.ParseDuration(b.config.RawBootWait)
b.config.bootWait, err = time.ParseDuration(b.config.RawBootWait)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing boot_wait: %s", err))
}
@@ -188,7 +233,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.RawShutdownTimeout = "5m"
}
b.config.ShutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
b.config.shutdownTimeout, err = time.ParseDuration(b.config.RawShutdownTimeout)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing shutdown_timeout: %s", err))
}
@@ -197,7 +242,7 @@ func (b *Builder) Prepare(raws ...interface{}) error {
b.config.RawSSHWaitTimeout = "20m"
}
b.config.SSHWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
b.config.sshWaitTimeout, err = time.ParseDuration(b.config.RawSSHWaitTimeout)
if err != nil {
errs = append(errs, fmt.Errorf("Failed parsing ssh_wait_timeout: %s", err))
}
@@ -230,17 +275,25 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
&stepPrepareTools{},
&stepDownloadISO{},
&stepPrepareOutputDir{},
&common.StepCreateFloppy{
Files: b.config.FloppyFiles,
},
&stepCreateDisk{},
&stepCreateVMX{},
&stepHTTPServer{},
&stepConfigureVNC{},
&stepRun{},
&stepTypeBootCommand{},
&stepWaitForSSH{},
&common.StepConnectSSH{
SSHAddress: sshAddress,
SSHConfig: sshConfig,
SSHWaitTimeout: b.config.sshWaitTimeout,
},
&stepUploadTools{},
&stepProvision{},
&common.StepProvision{},
&stepShutdown{},
&stepCleanFiles{},
&stepCleanVMX{},
&stepCompactDisk{},
}
+89 -19
View File
@@ -4,15 +4,17 @@ import (
"github.com/mitchellh/packer/packer"
"io/ioutil"
"os"
"reflect"
"testing"
"time"
)
func testConfig() map[string]interface{} {
return map[string]interface{}{
"iso_md5": "foo",
"iso_url": "http://www.packer.io",
"ssh_username": "foo",
"iso_checksum": "foo",
"iso_checksum_type": "md5",
"iso_url": "http://www.packer.io",
"ssh_username": "foo",
packer.BuildNameConfigKey: "foo",
}
@@ -56,6 +58,58 @@ func TestBuilderPrepare_BootWait(t *testing.T) {
}
}
func TestBuilderPrepare_ISOChecksum(t *testing.T) {
var b Builder
config := testConfig()
// Test bad
config["iso_checksum"] = ""
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test good
config["iso_checksum"] = "FOo"
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.ISOChecksum != "foo" {
t.Fatalf("should've lowercased: %s", b.config.ISOChecksum)
}
}
func TestBuilderPrepare_ISOChecksumType(t *testing.T) {
var b Builder
config := testConfig()
// Test bad
config["iso_checksum_type"] = ""
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test good
config["iso_checksum_type"] = "mD5"
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.ISOChecksumType != "md5" {
t.Fatalf("should've lowercased: %s", b.config.ISOChecksumType)
}
// Test unknown
config["iso_checksum_type"] = "fake"
err = b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_Defaults(t *testing.T) {
var b Builder
config := testConfig()
@@ -72,8 +126,8 @@ func TestBuilderPrepare_Defaults(t *testing.T) {
t.Errorf("bad output dir: %s", b.config.OutputDir)
}
if b.config.SSHWaitTimeout != (20 * time.Minute) {
t.Errorf("bad wait timeout: %s", b.config.SSHWaitTimeout)
if b.config.sshWaitTimeout != (20 * time.Minute) {
t.Errorf("bad wait timeout: %s", b.config.sshWaitTimeout)
}
if b.config.VMName != "packer-foo" {
@@ -107,6 +161,33 @@ func TestBuilderPrepare_DiskSize(t *testing.T) {
}
}
func TestBuilderPrepare_FloppyFiles(t *testing.T) {
var b Builder
config := testConfig()
delete(config, "floppy_files")
err := b.Prepare(config)
if err != nil {
t.Fatalf("bad err: %s", err)
}
if len(b.config.FloppyFiles) != 0 {
t.Fatalf("bad: %#v", b.config.FloppyFiles)
}
config["floppy_files"] = []string{"foo", "bar"}
b = Builder{}
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
expected := []string{"foo", "bar"}
if !reflect.DeepEqual(b.config.FloppyFiles, expected) {
t.Fatalf("bad: %#v", b.config.FloppyFiles)
}
}
func TestBuilderPrepare_HTTPPort(t *testing.T) {
var b Builder
config := testConfig()
@@ -135,27 +216,16 @@ func TestBuilderPrepare_HTTPPort(t *testing.T) {
}
}
func TestBuilderPrepare_ISOMD5(t *testing.T) {
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config := testConfig()
// Test bad
config["iso_md5"] = ""
// Add a random key
config["i_should_not_be_valid"] = true
err := b.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
// Test good
config["iso_md5"] = "FOo"
err = b.Prepare(config)
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.ISOMD5 != "foo" {
t.Fatalf("should've lowercased: %s", b.config.ISOMD5)
}
}
func TestBuilderPrepare_ISOUrl(t *testing.T) {
+5
View File
@@ -1,6 +1,7 @@
package vmware
import (
"errors"
"fmt"
"io/ioutil"
"os"
@@ -68,5 +69,9 @@ func (f *DHCPLeaseGuestLookup) GuestIP() (string, error) {
}
}
if curIp == "" {
return "", errors.New("IP not found for MAC in DHCP leases")
}
return curIp, nil
}
+70
View File
@@ -0,0 +1,70 @@
package vmware
import (
gossh "code.google.com/p/go.crypto/ssh"
"errors"
"fmt"
"github.com/mitchellh/packer/communicator/ssh"
"io/ioutil"
"log"
"os"
)
func sshAddress(state map[string]interface{}) (string, error) {
config := state["config"].(*config)
vmxPath := state["vmx_path"].(string)
log.Println("Lookup up IP information...")
f, err := os.Open(vmxPath)
if err != nil {
return "", err
}
defer f.Close()
vmxBytes, err := ioutil.ReadAll(f)
if err != nil {
return "", err
}
vmxData := ParseVMX(string(vmxBytes))
var ok bool
macAddress := ""
if macAddress, ok = vmxData["ethernet0.address"]; !ok || macAddress == "" {
if macAddress, ok = vmxData["ethernet0.generatedAddress"]; !ok || macAddress == "" {
return "", errors.New("couldn't find MAC address in VMX")
}
}
ipLookup := &DHCPLeaseGuestLookup{
Device: "vmnet8",
MACAddress: macAddress,
}
ipAddress, err := ipLookup.GuestIP()
if err != nil {
log.Printf("IP lookup failed: %s", err)
return "", fmt.Errorf("IP lookup failed: %s", err)
}
if ipAddress == "" {
log.Println("IP is blank, no IP yet.")
return "", errors.New("IP is blank")
}
log.Printf("Detected IP: %s", ipAddress)
return fmt.Sprintf("%s:%d", ipAddress, config.SSHPort), nil
}
func sshConfig(state map[string]interface{}) (*gossh.ClientConfig, error) {
config := state["config"].(*config)
return &gossh.ClientConfig{
User: config.SSHUser,
Auth: []gossh.ClientAuth{
gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
gossh.ClientAuthKeyboardInteractive(
ssh.PasswordKeyboardInteractive(config.SSHPassword)),
},
}, nil
}
+72
View File
@@ -0,0 +1,72 @@
package vmware
import (
"fmt"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/packer"
"io/ioutil"
"log"
"os"
"strings"
)
// This step cleans up the VMX by removing or changing this prior to
// being ready for use.
//
// Uses:
// ui packer.Ui
// vmx_path string
//
// Produces:
// <nothing>
type stepCleanVMX struct{}
func (s stepCleanVMX) Run(state map[string]interface{}) multistep.StepAction {
if _, ok := state["floppy_path"]; !ok {
return multistep.ActionContinue
}
ui := state["ui"].(packer.Ui)
vmxPath := state["vmx_path"].(string)
vmxData, err := s.readVMX(vmxPath)
if err != nil {
state["error"] = fmt.Errorf("Error reading VMX: %s", err)
return multistep.ActionHalt
}
// Delete the floppy0 entries so the floppy is no longer mounted
ui.Say("Unmounting floppy from VMX...")
for k, _ := range vmxData {
if strings.HasPrefix(k, "floppy0.") {
log.Printf("Deleting key: %s", k)
delete(vmxData, k)
}
}
vmxData["floppy0.present"] = "FALSE"
// Rewrite the VMX
if err := WriteVMX(vmxPath, vmxData); err != nil {
state["error"] = fmt.Errorf("Error writing VMX: %s", err)
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func (stepCleanVMX) Cleanup(map[string]interface{}) {}
func (stepCleanVMX) readVMX(vmxPath string) (map[string]string, error) {
vmxF, err := os.Open(vmxPath)
if err != nil {
return nil, err
}
defer vmxF.Close()
vmxBytes, err := ioutil.ReadAll(vmxF)
if err != nil {
return nil, err
}
return ParseVMX(string(vmxBytes)), nil
}
+11 -4
View File
@@ -36,10 +36,10 @@ func (stepCreateVMX) Run(state map[string]interface{}) multistep.StepAction {
ui.Say("Building and writing VMX file")
tplData := &vmxTemplateData{
config.VMName,
config.GuestOSType,
config.DiskName,
isoPath,
Name: config.VMName,
GuestOS: config.GuestOSType,
DiskName: config.DiskName,
ISOPath: isoPath,
}
var buf bytes.Buffer
@@ -55,6 +55,13 @@ func (stepCreateVMX) Run(state map[string]interface{}) multistep.StepAction {
}
}
if floppyPathRaw, ok := state["floppy_path"]; ok {
log.Println("Floppy path present, setting in VMX")
vmxData["floppy0.present"] = "TRUE"
vmxData["floppy0.fileType"] = "file"
vmxData["floppy0.fileName"] = floppyPathRaw.(string)
}
vmxPath := filepath.Join(config.OutputDir, config.VMName+".vmx")
if err := WriteVMX(vmxPath, vmxData); err != nil {
err := fmt.Errorf("Error creating VMX file: %s", err)
+2 -3
View File
@@ -1,7 +1,6 @@
package vmware
import (
"crypto/md5"
"encoding/hex"
"fmt"
"github.com/mitchellh/multistep"
@@ -27,7 +26,7 @@ func (s stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
checksum, err := hex.DecodeString(config.ISOMD5)
checksum, err := hex.DecodeString(config.ISOChecksum)
if err != nil {
err := fmt.Errorf("Error parsing checksum: %s", err)
state["error"] = err
@@ -43,7 +42,7 @@ func (s stepDownloadISO) Run(state map[string]interface{}) multistep.StepAction
Url: config.ISOUrl,
TargetPath: cachePath,
CopyFile: false,
Hash: md5.New(),
Hash: common.HashForType(config.ISOChecksumType),
Checksum: checksum,
}
@@ -10,6 +10,12 @@ type stepPrepareOutputDir struct{}
func (stepPrepareOutputDir) Run(state map[string]interface{}) multistep.StepAction {
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
if _, err := os.Stat(config.OutputDir); err == nil && config.PackerForce {
ui.Say("Deleting previous output directory...")
os.RemoveAll(config.OutputDir)
}
if err := os.MkdirAll(config.OutputDir, 0755); err != nil {
state["error"] = err
-25
View File
@@ -1,25 +0,0 @@
package vmware
import (
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/packer"
"log"
)
type stepProvision struct{}
func (*stepProvision) Run(state map[string]interface{}) multistep.StepAction {
comm := state["communicator"].(packer.Communicator)
hook := state["hook"].(packer.Hook)
ui := state["ui"].(packer.Ui)
log.Println("Running the provision hook")
if err := hook.Run(packer.HookProvision, ui, comm, nil); err != nil {
state["error"] = err
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func (*stepProvision) Cleanup(map[string]interface{}) {}
+3 -3
View File
@@ -49,9 +49,9 @@ func (s *stepRun) Run(state map[string]interface{}) multistep.StepAction {
}
// Wait the wait amount
if int64(config.BootWait) > 0 {
ui.Say(fmt.Sprintf("Waiting %s for boot...", config.BootWait.String()))
time.Sleep(config.BootWait)
if int64(config.bootWait) > 0 {
ui.Say(fmt.Sprintf("Waiting %s for boot...", config.bootWait.String()))
time.Sleep(config.bootWait)
}
return multistep.ActionContinue
+21 -3
View File
@@ -1,6 +1,7 @@
package vmware
import (
"bytes"
"errors"
"fmt"
"github.com/mitchellh/multistep"
@@ -35,7 +36,13 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
if config.ShutdownCommand != "" {
ui.Say("Gracefully halting virtual machine...")
log.Printf("Executing shutdown command: %s", config.ShutdownCommand)
cmd := &packer.RemoteCmd{Command: config.ShutdownCommand}
var stdout, stderr bytes.Buffer
cmd := &packer.RemoteCmd{
Command: config.ShutdownCommand,
Stdout: &stdout,
Stderr: &stderr,
}
if err := comm.Start(cmd); err != nil {
err := fmt.Errorf("Failed to send shutdown command: %s", err)
state["error"] = err
@@ -46,9 +53,20 @@ func (s *stepShutdown) Run(state map[string]interface{}) multistep.StepAction {
// Wait for the command to run
cmd.Wait()
// If the command failed to run, notify the user in some way.
if cmd.ExitStatus != 0 {
state["error"] = fmt.Errorf(
"Shutdown command has non-zero exit status.\n\nStdout: %s\n\nStderr: %s",
stdout.String(), stderr.String())
return multistep.ActionHalt
}
log.Printf("Shutdown stdout: %s", stdout.String())
log.Printf("Shutdown stderr: %s", stderr.String())
// Wait for the machine to actually shut down
log.Printf("Waiting max %s for shutdown to complete", config.ShutdownTimeout)
shutdownTimer := time.After(config.ShutdownTimeout)
log.Printf("Waiting max %s for shutdown to complete", config.shutdownTimeout)
shutdownTimer := time.After(config.shutdownTimeout)
for {
running, _ := driver.IsRunning(vmxPath)
if !running {
-185
View File
@@ -1,185 +0,0 @@
package vmware
import (
gossh "code.google.com/p/go.crypto/ssh"
"errors"
"fmt"
"github.com/mitchellh/multistep"
"github.com/mitchellh/packer/communicator/ssh"
"github.com/mitchellh/packer/packer"
"io/ioutil"
"log"
"net"
"os"
"time"
)
// This step waits for SSH to become available and establishes an SSH
// connection.
//
// Uses:
// config *config
// ui packer.Ui
// vmx_path string
//
// Produces:
// communicator packer.Communicator
type stepWaitForSSH struct {
cancel bool
conn net.Conn
}
func (s *stepWaitForSSH) Run(state map[string]interface{}) multistep.StepAction {
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
var comm packer.Communicator
var err error
waitDone := make(chan bool, 1)
go func() {
comm, err = s.waitForSSH(state)
waitDone <- true
}()
log.Printf("Waiting for SSH, up to timeout: %s", config.SSHWaitTimeout.String())
timeout := time.After(config.SSHWaitTimeout)
WaitLoop:
for {
// Wait for either SSH to become available, a timeout to occur,
// or an interrupt to come through.
select {
case <-waitDone:
if err != nil {
ui.Error(fmt.Sprintf("Error waiting for SSH: %s", err))
return multistep.ActionHalt
}
state["communicator"] = comm
break WaitLoop
case <-timeout:
ui.Error("Timeout waiting for SSH.")
s.cancel = true
return multistep.ActionHalt
case <-time.After(1 * time.Second):
if _, ok := state[multistep.StateCancelled]; ok {
log.Println("Interrupt detected, quitting waiting for SSH.")
return multistep.ActionHalt
}
}
}
return multistep.ActionContinue
}
func (s *stepWaitForSSH) Cleanup(map[string]interface{}) {
if s.conn != nil {
s.conn.Close()
s.conn = nil
}
}
// Reads the network information for lookup via DHCP.
func (s *stepWaitForSSH) dhcpLeaseLookup(vmxPath string) (GuestIPFinder, error) {
f, err := os.Open(vmxPath)
if err != nil {
return nil, err
}
defer f.Close()
vmxBytes, err := ioutil.ReadAll(f)
if err != nil {
return nil, err
}
vmxData := ParseVMX(string(vmxBytes))
var ok bool
macAddress := ""
if macAddress, ok = vmxData["ethernet0.address"]; !ok || macAddress == "" {
if macAddress, ok = vmxData["ethernet0.generatedAddress"]; !ok || macAddress == "" {
return nil, errors.New("couldn't find MAC address in VMX")
}
}
return &DHCPLeaseGuestLookup{"vmnet8", macAddress}, nil
}
// This blocks until SSH becomes available, and sends the communicator
// on the given channel.
func (s *stepWaitForSSH) waitForSSH(state map[string]interface{}) (packer.Communicator, error) {
config := state["config"].(*config)
ui := state["ui"].(packer.Ui)
vmxPath := state["vmx_path"].(string)
handshakeAttempts := 0
ui.Say("Waiting for SSH to become available...")
var comm packer.Communicator
var nc net.Conn
for {
if nc != nil {
nc.Close()
}
time.Sleep(5 * time.Second)
if s.cancel {
log.Println("SSH wait cancelled. Exiting loop.")
return nil, errors.New("SSH wait cancelled")
}
// First we wait for the IP to become available...
log.Println("Lookup up IP information...")
ipLookup, err := s.dhcpLeaseLookup(vmxPath)
if err != nil {
log.Printf("Can't lookup via DHCP lease: %s", err)
}
ip, err := ipLookup.GuestIP()
if err != nil {
log.Printf("IP lookup failed: %s", err)
continue
}
log.Printf("Detected IP: %s", ip)
// Attempt to connect to SSH port
nc, err = net.Dial("tcp", fmt.Sprintf("%s:%d", ip, config.SSHPort))
if err != nil {
log.Printf("TCP connection to SSH ip/port failed: %s", err)
continue
}
// Then we attempt to connect via SSH
sshConfig := &gossh.ClientConfig{
User: config.SSHUser,
Auth: []gossh.ClientAuth{
gossh.ClientAuthPassword(ssh.Password(config.SSHPassword)),
gossh.ClientAuthKeyboardInteractive(
ssh.PasswordKeyboardInteractive(config.SSHPassword)),
},
}
comm, err = ssh.New(nc, sshConfig)
if err != nil {
log.Printf("SSH handshake err: %s", err)
handshakeAttempts += 1
if handshakeAttempts < 10 {
// Try to connect via SSH a handful of times
continue
}
return nil, err
}
ui.Say("Connected via SSH!")
break
}
// Store the connection so we can close it later
s.conn = nc
return comm, nil
}
+5 -1
View File
@@ -21,12 +21,14 @@ func (Command) Help() string {
func (c Command) Run(env packer.Environment, args []string) int {
var cfgDebug bool
var cfgForce bool
var cfgExcept []string
var cfgOnly []string
cmdFlags := flag.NewFlagSet("build", flag.ContinueOnError)
cmdFlags.Usage = func() { env.Ui().Say(c.Help()) }
cmdFlags.BoolVar(&cfgDebug, "debug", false, "debug mode for builds")
cmdFlags.BoolVar(&cfgForce, "force", false, "force a build if artifacts exist")
cmdFlags.Var((*stringSliceValue)(&cfgExcept), "except", "build all builds except these")
cmdFlags.Var((*stringSliceValue)(&cfgOnly), "only", "only build the given builds by name")
if err := cmdFlags.Parse(args); err != nil {
@@ -141,11 +143,13 @@ func (c Command) Run(env packer.Environment, args []string) int {
env.Ui().Say("")
log.Printf("Build debug mode: %v", cfgDebug)
log.Printf("Force build: %v", cfgForce)
// Set the debug mode and prepare all the builds
// Set the debug and force mode and prepare all the builds
for _, b := range builds {
log.Printf("Preparing build: %s", b.Name())
b.SetDebug(cfgDebug)
b.SetForce(cfgForce)
err := b.Prepare()
if err != nil {
env.Ui().Error(err.Error())
+1
View File
@@ -9,6 +9,7 @@ Usage: packer build [options] TEMPLATE
Options:
-debug Debug mode enabled for builds
-force Force a build to continue if artifacts exist, deletes existing artifacts
-except=foo,bar,baz Build all builds other than these
-only=foo,bar,baz Only build the given builds by name
`
+85
View File
@@ -0,0 +1,85 @@
package fix
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"github.com/mitchellh/packer/packer"
"os"
"strings"
)
type Command byte
func (Command) Help() string {
return strings.TrimSpace(helpString)
}
func (c Command) Run(env packer.Environment, args []string) int {
cmdFlags := flag.NewFlagSet("fix", flag.ContinueOnError)
cmdFlags.Usage = func() { env.Ui().Say(c.Help()) }
if err := cmdFlags.Parse(args); err != nil {
return 1
}
args = cmdFlags.Args()
if len(args) != 1 {
cmdFlags.Usage()
return 1
}
// Read the file for decoding
tplF, err := os.Open(args[0])
if err != nil {
env.Ui().Error(fmt.Sprintf("Error opening template: %s", err))
return 1
}
defer tplF.Close()
// Decode the JSON into a generic map structure
var templateData map[string]interface{}
decoder := json.NewDecoder(tplF)
if err := decoder.Decode(&templateData); err != nil {
env.Ui().Error(fmt.Sprintf("Error parsing template: %s", err))
return 1
}
// Close the file since we're done with that
tplF.Close()
// Run the template through the various fixers
fixers := []Fixer{Fixers["iso-md5"]}
input := templateData
for _, fixer := range fixers {
var err error
input, err = fixer.Fix(input)
if err != nil {
env.Ui().Error(fmt.Sprintf("Error fixing: %s", err))
return 1
}
}
var output bytes.Buffer
encoder := json.NewEncoder(&output)
if err := encoder.Encode(input); err != nil {
env.Ui().Error(fmt.Sprintf("Error encoding: %s", err))
return 1
}
var indented bytes.Buffer
if err := json.Indent(&indented, output.Bytes(), "", " "); err != nil {
env.Ui().Error(fmt.Sprintf("Error encoding: %s", err))
return 1
}
result := indented.String()
result = strings.Replace(result, `\u003c`, "<", -1)
result = strings.Replace(result, `\u003e`, ">", -1)
env.Ui().Say(result)
return 0
}
func (c Command) Synopsis() string {
return "fixes templates from old versions of packer"
}
+14
View File
@@ -0,0 +1,14 @@
package fix
import (
"github.com/mitchellh/packer/packer"
"testing"
)
func TestCommand_Impl(t *testing.T) {
var raw interface{}
raw = new(Command)
if _, ok := raw.(packer.Command); !ok {
t.Fatalf("must be a Command")
}
}
+18
View File
@@ -0,0 +1,18 @@
package fix
// A Fixer is something that can perform a fix operation on a template.
type Fixer interface {
// Fix takes a raw map structure input, potentially transforms it
// in some way, and returns the new, transformed structure. The
// Fix method is allowed to mutate the input.
Fix(input map[string]interface{}) (map[string]interface{}, error)
}
// Fixers is the map of all available fixers, by name.
var Fixers map[string]Fixer
func init() {
Fixers = map[string]Fixer{
"iso-md5": new(FixerISOMD5),
}
}
+43
View File
@@ -0,0 +1,43 @@
package fix
import (
"github.com/mitchellh/mapstructure"
)
// FixerISOMD5 is a Fixer that replaces the "iso_md5" configuration key
// with the newer "iso_checksum" and "iso_checksum_type" within builders.
type FixerISOMD5 struct{}
func (FixerISOMD5) Fix(input map[string]interface{}) (map[string]interface{}, error) {
// Our template type we'll use for this fixer only
type template struct {
Builders []map[string]interface{}
}
// Decode the input into our structure, if we can
var tpl template
if err := mapstructure.Decode(input, &tpl); err != nil {
return nil, err
}
// Go through each builder and replace the iso_md5 if we can
for _, builder := range tpl.Builders {
md5raw, ok := builder["iso_md5"]
if !ok {
continue
}
md5, ok := md5raw.(string)
if !ok {
// TODO: error?
continue
}
delete(builder, "iso_md5")
builder["iso_checksum"] = md5
builder["iso_checksum_type"] = "md5"
}
input["builders"] = tpl.Builders
return input, nil
}
+46
View File
@@ -0,0 +1,46 @@
package fix
import (
"reflect"
"testing"
)
func TestFixerISOMD5_Impl(t *testing.T) {
var raw interface{}
raw = new(FixerISOMD5)
if _, ok := raw.(Fixer); !ok {
t.Fatalf("must be a Fixer")
}
}
func TestFixerISOMD5_Fix(t *testing.T) {
var f FixerISOMD5
input := map[string]interface{}{
"builders": []interface{}{
map[string]string{
"type": "foo",
"iso_md5": "bar",
},
},
}
expected := map[string]interface{}{
"builders": []map[string]interface{}{
map[string]interface{}{
"type": "foo",
"iso_checksum": "bar",
"iso_checksum_type": "md5",
},
},
}
output, err := f.Fix(input)
if err != nil {
t.Fatalf("err: %s", err)
}
if !reflect.DeepEqual(output, expected) {
t.Fatalf("unexpected: %#v\nexpected: %#v\n", output, expected)
}
}
+16
View File
@@ -0,0 +1,16 @@
package fix
const helpString = `
Usage: packer fix [options] TEMPLATE
Reads the JSON template and attempts to fix known backwards
incompatibilities. The fixed template will be outputted to standard out.
If the template cannot be fixed due to an error, the command will exit
with a non-zero exit status. Error messages will appear on standard error.
Fixes that are run:
iso-md5 Replaces "iso_md5" in builders with newer "iso_checksum"
`
+62 -7
View File
@@ -14,18 +14,39 @@ import (
type comm struct {
client *ssh.ClientConn
config *Config
conn net.Conn
}
// Config is the structure used to configure the SSH communicator.
type Config struct {
// The configuration of the Go SSH connection
SSHConfig *ssh.ClientConfig
// Connection returns a new connection. The current connection
// in use will be closed as part of the Close method, or in the
// case an error occurs.
Connection func() (net.Conn, error)
}
// Creates a new packer.Communicator implementation over SSH. This takes
// an already existing TCP connection and SSH configuration.
func New(c net.Conn, config *ssh.ClientConfig) (result *comm, err error) {
client, err := ssh.Client(c, config)
result = &comm{client}
func New(config *Config) (result *comm, err error) {
// Establish an initial connection and connect
result = &comm{
config: config,
}
if err = result.reconnect(); err != nil {
result = nil
return
}
return
}
func (c *comm) Start(cmd *packer.RemoteCmd) (err error) {
session, err := c.client.NewSession()
session, err := c.newSession()
if err != nil {
return
}
@@ -56,7 +77,6 @@ func (c *comm) Start(cmd *packer.RemoteCmd) (err error) {
// exit boolean and status.
go func() {
defer session.Close()
err := session.Wait()
cmd.ExitStatus = 0
if err != nil {
@@ -73,8 +93,7 @@ func (c *comm) Start(cmd *packer.RemoteCmd) (err error) {
}
func (c *comm) Upload(path string, input io.Reader) error {
log.Println("Opening new SSH session")
session, err := c.client.NewSession()
session, err := c.newSession()
if err != nil {
return err
}
@@ -168,3 +187,39 @@ func (c *comm) Upload(path string, input io.Reader) error {
func (c *comm) Download(string, io.Writer) error {
panic("not implemented yet")
}
func (c *comm) newSession() (*ssh.Session, error) {
log.Println("opening new ssh session")
session, err := c.client.NewSession()
if err != nil {
log.Printf("ssh session open error: '%s', attempting reconnect", err)
if err := c.reconnect(); err != nil {
return nil, err
}
return c.client.NewSession()
}
return session, nil
}
func (c *comm) reconnect() (err error) {
if c.conn != nil {
c.conn.Close()
}
log.Printf("reconnecting to TCP connection for SSH")
c.conn, err = c.config.Connection()
if err != nil {
log.Printf("reconnection error: %s", err)
return
}
log.Printf("handshaking with SSH")
c.client, err = ssh.Client(c.conn, c.config.SSHConfig)
if err != nil {
log.Printf("handshake error: %s", err)
}
return
}
+24 -8
View File
@@ -115,12 +115,20 @@ func TestNew_Invalid(t *testing.T) {
},
}
conn, err := net.Dial("tcp", newMockLineServer(t))
if err != nil {
t.Fatalf("unable to dial to remote side: %s", err)
conn := func() (net.Conn, error) {
conn, err := net.Dial("tcp", newMockLineServer(t))
if err != nil {
t.Fatalf("unable to dial to remote side: %s", err)
}
return conn, err
}
_, err = New(conn, clientConfig)
config := &Config{
Connection: conn,
SSHConfig: clientConfig,
}
_, err := New(config)
if err == nil {
t.Fatal("should have had an error connecting")
}
@@ -134,12 +142,20 @@ func TestStart(t *testing.T) {
},
}
conn, err := net.Dial("tcp", newMockLineServer(t))
if err != nil {
t.Fatalf("unable to dial to remote side: %s", err)
conn := func() (net.Conn, error) {
conn, err := net.Dial("tcp", newMockLineServer(t))
if err != nil {
t.Fatalf("unable to dial to remote side: %s", err)
}
return conn, err
}
client, err := New(conn, clientConfig)
config := &Config{
Connection: conn,
SSHConfig: clientConfig,
}
client, err := New(config)
if err != nil {
t.Fatalf("error connecting to SSH: %s", err)
}
+33
View File
@@ -0,0 +1,33 @@
package ssh
import (
"errors"
"log"
"net"
"time"
)
// ConnectFunc is a convenience method for returning a function
// that just uses net.Dial to communicate with the remote end that
// is suitable for use with the SSH communicator configuration.
func ConnectFunc(network, addr string, timeout time.Duration) func() (net.Conn, error) {
return func() (net.Conn, error) {
timeoutCh := time.After(timeout)
for {
select {
case <-timeoutCh:
return nil, errors.New("timeout connecting to remote machine")
default:
}
log.Printf("Opening conn for SSH to %s %s", network, addr)
nc, err := net.DialTimeout(network, addr, 15*time.Second)
if err == nil {
return nc, nil
}
time.Sleep(500 * time.Millisecond)
}
}
}
+1
View File
@@ -27,6 +27,7 @@ const defaultConfig = `
"commands": {
"build": "packer-command-build",
"fix": "packer-command-fix",
"validate": "packer-command-validate"
},
+24 -8
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"github.com/mitchellh/packer/packer"
"github.com/mitchellh/packer/packer/plugin"
"io"
"io/ioutil"
"log"
"os"
@@ -14,20 +15,35 @@ import (
)
func main() {
if os.Getenv("PACKER_LOG") == "" {
// If we don't have logging explicitly enabled, then disable it
log.SetOutput(ioutil.Discard)
} else {
// Logging is enabled, make sure it goes to stderr
log.SetOutput(os.Stderr)
// Setup logging if PACKER_LOG is set.
// Log to PACKER_LOG_PATH if it is set, otherwise default to stderr.
var logOutput io.Writer = ioutil.Discard
if os.Getenv("PACKER_LOG") != "" {
logOutput = os.Stderr
if logPath := os.Getenv("PACKER_LOG_PATH"); logPath != "" {
var err error
logOutput, err = os.Create(logPath)
if err != nil {
fmt.Fprintf(
os.Stderr,
"Couldn't open '%s' for logging: %s",
logPath, err)
os.Exit(1)
}
}
}
log.SetOutput(logOutput)
// If there is no explicit number of Go threads to use, then set it
if os.Getenv("GOMAXPROCS") == "" {
runtime.GOMAXPROCS(runtime.NumCPU())
}
log.Printf("Packer Version: %s %s", packer.Version, packer.VersionPrerelease)
log.Printf(
"Packer Version: %s %s %s",
packer.Version, packer.VersionPrerelease, packer.GitCommit)
log.Printf("Packer Target OS/Arch: %s %s", runtime.GOOS, runtime.GOARCH)
config, err := loadConfig()
@@ -100,7 +116,7 @@ func loadConfig() (*config, error) {
mustExist = false
if err != nil {
log.Printf("Error detecing default config file path: %s", err)
log.Printf("Error detecting default config file path: %s", err)
}
}
+36 -8
View File
@@ -6,13 +6,24 @@ import (
"sync"
)
// This is the key in configurations that is set to the name of the
// build.
const BuildNameConfigKey = "packer_build_name"
const (
// This is the key in configurations that is set to the name of the
// build.
BuildNameConfigKey = "packer_build_name"
// This is the key in configurations that is set to "true" when Packer
// debugging is enabled.
const DebugConfigKey = "packer_debug"
// This is the key in the configuration that is set to the type
// of the builder that is run. This is useful for provisioners and
// such who want to make use of this.
BuilderTypeConfigKey = "packer_builder_type"
// This is the key in configurations that is set to "true" when Packer
// debugging is enabled.
DebugConfigKey = "packer_debug"
// This is the key in configurations that is set to "true" when Packer
// force build is enabled.
ForceConfigKey = "packer_force"
)
// A Build represents a single job within Packer that is responsible for
// building some machine image artifact. Builds are meant to be parallelized.
@@ -42,6 +53,12 @@ type Build interface {
// When SetDebug is set to true, parallelism between builds is
// strictly prohibited.
SetDebug(bool)
// SetForce will enable/disable forcing a build when artifacts exist.
//
// When SetForce is set to true, existing artifacts from the build are
// deleted prior to the build.
SetForce(bool)
}
// A build struct represents a single build job, the result of which should
@@ -58,6 +75,7 @@ type coreBuild struct {
provisioners []coreBuildProvisioner
debug bool
force bool
l sync.Mutex
prepareCalled bool
}
@@ -96,8 +114,10 @@ func (b *coreBuild) Prepare() (err error) {
b.prepareCalled = true
packerConfig := map[string]interface{}{
BuildNameConfigKey: b.name,
DebugConfigKey: b.debug,
BuildNameConfigKey: b.name,
BuilderTypeConfigKey: b.builderType,
DebugConfigKey: b.debug,
ForceConfigKey: b.force,
}
// Prepare the builder
@@ -265,6 +285,14 @@ func (b *coreBuild) SetDebug(val bool) {
b.debug = val
}
func (b *coreBuild) SetForce(val bool) {
if b.prepareCalled {
panic("prepare has already been called")
}
b.force = val
}
// Cancels the build if it is running.
func (b *coreBuild) Cancel() {
b.builder.Cancel()
+9 -4
View File
@@ -11,6 +11,7 @@ func testBuild() *coreBuild {
name: "test",
builder: &TestBuilder{artifactId: "b"},
builderConfig: 42,
builderType: "foo",
hooks: map[string][]Hook{
"foo": []Hook{&TestHook{}},
},
@@ -40,8 +41,10 @@ func TestBuild_Prepare(t *testing.T) {
assert := asserts.NewTestingAsserts(t, true)
packerConfig := map[string]interface{}{
BuildNameConfigKey: "test",
DebugConfigKey: false,
BuildNameConfigKey: "test",
BuilderTypeConfigKey: "foo",
DebugConfigKey: false,
ForceConfigKey: false,
}
build := testBuild()
@@ -86,8 +89,10 @@ func TestBuild_Prepare_Debug(t *testing.T) {
assert := asserts.NewTestingAsserts(t, true)
packerConfig := map[string]interface{}{
BuildNameConfigKey: "test",
DebugConfigKey: true,
BuildNameConfigKey: "test",
BuilderTypeConfigKey: "foo",
DebugConfigKey: true,
ForceConfigKey: false,
}
build := testBuild()
+11
View File
@@ -69,6 +69,12 @@ func (b *build) SetDebug(val bool) {
}
}
func (b *build) SetForce(val bool) {
if err := b.client.Call("Build.SetForce", val, new(interface{})); err != nil {
panic(err)
}
}
func (b *build) Cancel() {
if err := b.client.Call("Build.Cancel", new(interface{}), new(interface{})); err != nil {
panic(err)
@@ -111,6 +117,11 @@ func (b *BuildServer) SetDebug(val *bool, reply *interface{}) error {
return nil
}
func (b *BuildServer) SetForce(val *bool, reply *interface{}) error {
b.build.SetForce(*val)
return nil
}
func (b *BuildServer) Cancel(args *interface{}, reply *interface{}) error {
b.build.Cancel()
return nil
+9
View File
@@ -17,6 +17,7 @@ type testBuild struct {
runCache packer.Cache
runUi packer.Ui
setDebugCalled bool
setForceCalled bool
cancelCalled bool
errRunResult bool
@@ -48,6 +49,10 @@ func (b *testBuild) SetDebug(bool) {
b.setDebugCalled = true
}
func (b *testBuild) SetForce(bool) {
b.setForceCalled = true
}
func (b *testBuild) Cancel() {
b.cancelCalled = true
}
@@ -104,6 +109,10 @@ func TestBuildRPC(t *testing.T) {
bClient.SetDebug(true)
assert.True(b.setDebugCalled, "should be called")
// Test SetForce
bClient.SetForce(true)
assert.True(b.setForceCalled, "should be called")
// Test Cancel
bClient.Cancel()
assert.True(b.cancelCalled, "cancel should be called")
+39 -5
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"github.com/mitchellh/mapstructure"
"sort"
)
// The rawTemplate struct represents the structure of a template read
@@ -15,7 +16,7 @@ type rawTemplate struct {
Builders []map[string]interface{}
Hooks map[string][]string
Provisioners []map[string]interface{}
PostProcessors []interface{} `json:"post-processors"`
PostProcessors []interface{} `mapstructure:"post-processors"`
}
// The Template struct represents a parsed template, parsed into the most
@@ -63,8 +64,8 @@ type rawProvisionerConfig struct {
// and checking for this can be useful, if you wish to format it in a certain
// way.
func ParseTemplate(data []byte) (t *Template, err error) {
var rawTpl rawTemplate
err = json.Unmarshal(data, &rawTpl)
var rawTplInterface interface{}
err = json.Unmarshal(data, &rawTplInterface)
if err != nil {
syntaxErr, ok := err.(*json.SyntaxError)
if !ok {
@@ -92,14 +93,41 @@ func ParseTemplate(data []byte) (t *Template, err error) {
return
}
// Decode the raw template interface into the actual rawTemplate
// structure, checking for any extranneous keys along the way.
var md mapstructure.Metadata
var rawTpl rawTemplate
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &rawTpl,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return
}
err = decoder.Decode(rawTplInterface)
if err != nil {
return
}
errors := make([]error, 0)
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
errors = append(
errors, fmt.Errorf("Unknown root level key in template: '%s'", unused))
}
}
t = &Template{}
t.Builders = make(map[string]rawBuilderConfig)
t.Hooks = rawTpl.Hooks
t.PostProcessors = make([][]rawPostProcessorConfig, len(rawTpl.PostProcessors))
t.Provisioners = make([]rawProvisionerConfig, len(rawTpl.Provisioners))
errors := make([]error, 0)
// Gather all the builders
for i, v := range rawTpl.Builders {
var raw rawBuilderConfig
@@ -193,6 +221,11 @@ func ParseTemplate(data []byte) (t *Template, err error) {
continue
}
// The provisioners not only don't need or want the override settings
// (as they are processed as part of the preparation below), but will
// actively reject them as invalid configuration.
delete(v, "override")
raw.rawConfig = v
}
@@ -203,6 +236,7 @@ func ParseTemplate(data []byte) (t *Template, err error) {
// If there were errors, we put it into a MultiError and return
if len(errors) > 0 {
err = &MultiError{errors}
t = nil
return
}
+34 -1
View File
@@ -2,6 +2,7 @@ package packer
import (
"cgl.tideland.biz/asserts"
"reflect"
"sort"
"testing"
)
@@ -37,6 +38,23 @@ func TestParseTemplate_Invalid(t *testing.T) {
assert.Nil(result, "should have no result")
}
func TestParseTemplate_InvalidKeys(t *testing.T) {
assert := asserts.NewTestingAsserts(t, true)
// Note there is an extra comma below for a purposeful
// syntax error in the JSON.
data := `
{
"builders": [{"type": "foo"}],
"what is this": ""
}
`
result, err := ParseTemplate([]byte(data))
assert.NotNil(err, "should have an error")
assert.Nil(result, "should have no result")
}
func TestParseTemplate_BuilderWithoutType(t *testing.T) {
assert := asserts.NewTestingAsserts(t, true)
@@ -539,7 +557,22 @@ func TestTemplate_Build_ProvisionerOverride(t *testing.T) {
`
template, err := ParseTemplate([]byte(data))
assert.Nil(err, "should not error")
if err != nil {
t.Fatalf("err: %s", err)
}
rawConfig := template.Provisioners[0].rawConfig
if rawConfig == nil {
t.Fatal("missing provisioner raw config")
}
expected := map[string]interface{}{
"type": "test-prov",
}
if !reflect.DeepEqual(rawConfig, expected) {
t.Fatalf("bad raw: %#v", rawConfig)
}
builder := testBuilder()
builderMap := map[string]Builder{
+2 -1
View File
@@ -10,6 +10,7 @@ import (
"os/signal"
"strings"
"sync"
"unicode"
)
type UiColor uint
@@ -110,7 +111,7 @@ func (u *PrefixedUi) prefixLines(prefix, message string) string {
result.WriteString(fmt.Sprintf("%s: %s\n", prefix, line))
}
return strings.TrimSpace(result.String())
return strings.TrimRightFunc(result.String(), unicode.IsSpace)
}
func (rw *ReaderWriterUi) Ask(query string) (string, error) {
+9 -1
View File
@@ -5,8 +5,12 @@ import (
"fmt"
)
// The git commit that is being compiled. This will be filled in by the
// compiler for source builds.
var GitCommit string
// The version of packer.
const Version = "0.1.5"
const Version = "0.2.0"
// Any pre-release marker for the version. If this is "" (empty string),
// then it means that it is a final release. Otherwise, this is the
@@ -27,6 +31,10 @@ func (versionCommand) Run(env Environment, args []string) int {
fmt.Fprintf(&versionString, "Packer v%s", Version)
if VersionPrerelease != "" {
fmt.Fprintf(&versionString, ".%s", VersionPrerelease)
if GitCommit != "" {
fmt.Fprintf(&versionString, " (%s)", GitCommit)
}
}
env.Ui().Say(versionString.String())
+2 -2
View File
@@ -1,10 +1,10 @@
package main
import (
"github.com/mitchellh/packer/builder/amazonebs"
"github.com/mitchellh/packer/builder/amazon/ebs"
"github.com/mitchellh/packer/packer/plugin"
)
func main() {
plugin.ServeBuilder(new(amazonebs.Builder))
plugin.ServeBuilder(new(ebs.Builder))
}
+10
View File
@@ -0,0 +1,10 @@
package main
import (
"github.com/mitchellh/packer/command/fix"
"github.com/mitchellh/packer/packer/plugin"
)
func main() {
plugin.ServeCommand(new(fix.Command))
}
+28 -2
View File
@@ -6,6 +6,8 @@ import (
"github.com/mitchellh/mapstructure"
"github.com/mitchellh/packer/packer"
"os"
"sort"
"strings"
)
type config struct {
@@ -21,13 +23,37 @@ type Provisioner struct {
}
func (p *Provisioner) Prepare(raws ...interface{}) error {
var md mapstructure.Metadata
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &p.config,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return err
}
for _, raw := range raws {
if err := mapstructure.Decode(raw, &p.config); err != nil {
err := decoder.Decode(raw)
if err != nil {
return err
}
}
errs := []error{}
// Accumulate any errors
errs := make([]error, 0)
// Unused keys are errors
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
if unused != "type" && !strings.HasPrefix(unused, "packer_") {
errs = append(
errs, fmt.Errorf("Unknown configuration key: %s", unused))
}
}
}
if _, err := os.Stat(p.config.Source); err != nil {
errs = append(errs,
+12
View File
@@ -23,6 +23,18 @@ func TestProvisioner_Impl(t *testing.T) {
}
}
func TestProvisionerPrepare_InvalidKey(t *testing.T) {
var p Provisioner
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
err := p.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
func TestProvisionerPrepare_InvalidSource(t *testing.T) {
var p Provisioner
config := testConfig()
+42 -5
View File
@@ -14,6 +14,7 @@ import (
"io/ioutil"
"log"
"os"
"sort"
"strings"
"text/template"
)
@@ -46,6 +47,10 @@ type config struct {
// should be used to specify where the script goes, {{ .Vars }}
// can be used to inject the environment_vars into the environment.
ExecuteCommand string `mapstructure:"execute_command"`
// Packer configurations, these come from Packer itself
PackerBuildName string `mapstructure:"packer_build_name"`
PackerBuilderType string `mapstructure:"packer_builder_type"`
}
type Provisioner struct {
@@ -58,12 +63,38 @@ type ExecuteCommandTemplate struct {
}
func (p *Provisioner) Prepare(raws ...interface{}) error {
var md mapstructure.Metadata
decoderConfig := &mapstructure.DecoderConfig{
Metadata: &md,
Result: &p.config,
}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
return err
}
for _, raw := range raws {
if err := mapstructure.Decode(raw, &p.config); err != nil {
err := decoder.Decode(raw)
if err != nil {
return err
}
}
// Accumulate any errors
errs := make([]error, 0)
// Unused keys are errors
if len(md.Unused) > 0 {
sort.Strings(md.Unused)
for _, unused := range md.Unused {
if unused != "type" && !strings.HasPrefix(unused, "packer_") {
errs = append(
errs, fmt.Errorf("Unknown configuration key: %s", unused))
}
}
}
if p.config.ExecuteCommand == "" {
p.config.ExecuteCommand = "chmod +x {{.Path}}; {{.Vars}} {{.Path}}"
}
@@ -88,8 +119,6 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
p.config.Vars = make([]string, 0)
}
errs := make([]error, 0)
if p.config.Script != "" && len(p.config.Scripts) > 0 {
errs = append(errs, errors.New("Only one of script or scripts can be specified."))
}
@@ -114,7 +143,9 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
for _, kv := range p.config.Vars {
vs := strings.Split(kv, "=")
if len(vs) != 2 || vs[0] == "" {
errs = append(errs, fmt.Errorf("Environment variable not in format 'key=value': %s", kv))
errs = append(
errs,
fmt.Errorf("Environment variable not in format 'key=value': %s", kv))
}
}
@@ -157,6 +188,12 @@ func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
tf.Close()
}
// Build our variables up by adding in the build name and builder type
envVars := make([]string, len(p.config.Vars)+2)
envVars[0] = "PACKER_BUILD_NAME=" + p.config.PackerBuildName
envVars[1] = "PACKER_BUILDER_TYPE=" + p.config.PackerBuilderType
copy(envVars[2:], p.config.Vars)
for _, path := range scripts {
ui.Say(fmt.Sprintf("Provisioning with shell script: %s", path))
@@ -177,7 +214,7 @@ func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
f.Close()
// Flatten the environment variables
flattendVars := strings.Join(p.config.Vars, " ")
flattendVars := strings.Join(envVars, " ")
// Compile the command
var command bytes.Buffer
+12
View File
@@ -62,6 +62,18 @@ func TestProvisionerPrepare_InlineShebang(t *testing.T) {
}
}
func TestProvisionerPrepare_InvalidKey(t *testing.T) {
var p Provisioner
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
err := p.Prepare(config)
if err == nil {
t.Fatal("should have error")
}
}
func TestProvisionerPrepare_Script(t *testing.T) {
config := testConfig()
delete(config, "inline")
+14 -2
View File
@@ -1,4 +1,6 @@
#!/bin/bash
#
# This script only builds the application from source.
set -e
NO_COLOR="\x1b[0m"
@@ -14,13 +16,23 @@ DIR="$( cd -P "$( dirname "$SOURCE" )/.." && pwd )"
# Change into that directory
cd $DIR
# Get the git commit
GIT_COMMIT=$(git rev-parse --short HEAD)
GIT_DIRTY=$(test -n "`git status --porcelain`" && echo "+CHANGES" || true)
# Compile the main Packer app
echo -e "${OK_COLOR}--> Compiling Packer${NO_COLOR}"
go build -v -o bin/packer .
go build \
-ldflags "-X github.com/mitchellh/packer/packer.GitCommit ${GIT_COMMIT}${GIT_DIRTY}" \
-v \
-o bin/packer .
# Go over each plugin and build it
for PLUGIN in $(find ./plugin -mindepth 1 -maxdepth 1 -type d); do
PLUGIN_NAME=$(basename ${PLUGIN})
echo -e "${OK_COLOR}--> Compiling Plugin: ${PLUGIN_NAME}${NO_COLOR}"
go build -v -o bin/packer-${PLUGIN_NAME} ${PLUGIN}
go build \
-ldflags "-X github.com/mitchellh/packer/packer.GitCommit ${GIT_COMMIT}${GIT_DIRTY}" \
-v \
-o bin/packer-${PLUGIN_NAME} ${PLUGIN}
done
@@ -63,6 +63,12 @@ Required:
Optional:
* `security_group_id` (string) - The ID (_not_ the name) of the security
group to assign to the instance. By default this is not set and Packer
will automatically create a new temporary security group to allow SSH
access. Note that if this is specified, you must be sure the security
group allows access to the `ssh_port` given below.
* `ssh_port` (int) - The port that SSH will be available on. This defaults
to port 22.
@@ -25,11 +25,13 @@ Required:
* `api_key` (string) - The API key to use to access your account. You can
retrieve this on the "API" page visible after logging into your account
on DigitalOcean.
on DigitalOcean. Alternatively, the builder looks for the environment
variable `DIGITALOCEAN_API_KEY`.
* `client_id` (string) - The client ID to use to access your account. You can
find this on the "API" page visible after logging into your account on
DigitalOcean.
DigitalOcean. Alternatively, the builder looks for the environment
variable `DIGITALOCEAN_CLIENT_ID`.
Optional:
@@ -25,7 +25,8 @@ Ubuntu to self-install. Still, the example serves to show the basic configuratio
"type": "virtualbox",
"guest_os_type": "Ubuntu_64",
"iso_url": "http://releases.ubuntu.com/12.04/ubuntu-12.04.2-server-amd64.iso",
"iso_md5": "af5f788aee1b32c4b2634734309cc9e9",
"iso_checksum": "af5f788aee1b32c4b2634734309cc9e9",
"iso_checksum_type": "md5",
"ssh_username": "packer",
"ssh_wait_timeout": "30s",
"shutdown_command": "shutdown -P now"
@@ -40,9 +41,13 @@ each category, the available options are alphabetized and described.
Required:
* `iso_md5` (string) - The MD5 checksum for the OS ISO file. Because ISO
* `iso_checksum` (string) - The checksum for the OS ISO file. Because ISO
files are so large, this is required and Packer will verify it prior
to booting a virtual machine with the ISO attached.
to booting a virtual machine with the ISO attached. The type of the
checksum is specified with `iso_checksum_type`, documented below.
* `iso_checksum_type` (string) - The type of the checksum specified in
`iso_checksum`. Valid values are "md5", "sha1", or "sha256" currently.
* `iso_url` (string) - A URL to the ISO containing the installation image.
This URL can be either an HTTP URL or a file URL (or path to a file).
@@ -70,6 +75,13 @@ Optional:
* `disk_size` (int) - The size, in megabytes, of the hard disk to create
for the VM. By default, this is 40000 (40 GB).
* `floppy_files` (array of strings) - A list of files to put onto a floppy
disk that is attached when the VM is booted for the first time. This is
most useful for unattended Windows installs, which look for an
`Autounattend.xml` file on removable media. By default no floppy will
be attached. The files listed in this configuration will all be put
into the root directory of the floppy disk; sub-directories are not supported.
* `guest_additions_path` (string) - The path on the guest virtual machine
where the VirtualBox guest additions ISO will be uploaded. By default this
is "VBoxGuestAdditions.iso" which should upload into the login directory
@@ -26,7 +26,8 @@ Ubuntu to self-install. Still, the example serves to show the basic configuratio
{
"type": "vmware",
"iso_url": "http://releases.ubuntu.com/12.04/ubuntu-12.04.2-server-amd64.iso",
"iso_md5": "af5f788aee1b32c4b2634734309cc9e9",
"iso_checksum": "af5f788aee1b32c4b2634734309cc9e9",
"iso_checksum_type": "md5",
"ssh_username": "packer",
"ssh_wait_timeout": "30s",
"shutdown_command": "shutdown -P now"
@@ -41,9 +42,13 @@ each category, the available options are alphabetized and described.
Required:
* `iso_md5` (string) - The MD5 checksum for the OS ISO file. Because ISO
* `iso_checksum` (string) - The checksum for the OS ISO file. Because ISO
files are so large, this is required and Packer will verify it prior
to booting a virtual machine with the ISO attached.
to booting a virtual machine with the ISO attached. The type of the
checksum is specified with `iso_checksum_type`, documented below.
* `iso_checksum_type` (string) - The type of the checksum specified in
`iso_checksum`. Valid values are "md5", "sha1", or "sha256" currently.
* `iso_url` (string) - A URL to the ISO containing the installation image.
This URL can be either an HTTP URL or a file URL (or path to a file).
@@ -73,6 +78,13 @@ Optional:
actual file representing the disk will not use the full size unless it is full.
By default this is set to 40,000 (40 GB).
* `floppy_files` (array of strings) - A list of files to put onto a floppy
disk that is attached when the VM is booted for the first time. This is
most useful for unattended Windows installs, which look for an
`Autounattend.xml` file on removable media. By default no floppy will
be attached. The files listed in this configuration will all be put
into the root directory of the floppy disk; sub-directories are not supported.
* `guest_os_type` (string) - The guest OS type being installed. This will be
set in the VMware VMX. By default this is "other". By specifying a more specific
OS type, VMware may perform some optimizations or virtual hardware changes
@@ -17,6 +17,12 @@ artifacts that are created will be outputted at the end of the build.
between each step, waiting for keyboard input before continuing. This will allow
the user to inspect state and so on.
* `-force` - Forces a builder to run when artifacts from a previous build prevent
a build from running. The exact behavior of a forced build is left to the builder.
In general, a builder supporting the forced build will remove the artifacts from
the previous build. This will allow the user to repeat a build without having to
manually clean these artifacts beforehand.
* `-except=foo,bar,baz` - Builds all the builds except those with the given
comma-separated names. Build names by default are the names of their builders,
unless a specific `name` attribute is specified within the configuration.
@@ -0,0 +1,33 @@
---
layout: "docs"
page_title: "Command-line: Fix"
---
# Command-Line: Fix
The `packer fix` command takes a template and finds backwards incompatible
parts of it and brings it up to date so it can be used with the latest version
of Packer. After you update to a new Packer release, you should run the
fix command to make sure your templates work with the new release.
The fix command will output the changed template to standard out, so you
should redirect standard using standard OS-specific techniques if you want to
save it to a file. For example, on Linux systems, you may want to do this:
```
$ packer fix old.json > new.json
```
If fixing fails for any reason, the fix command will exit with a non-zero
exit status. Error messages appear on standard error, so if you're redirecting
output, you'll still see error messages.
<div class="alert alert-block alert-info">
<strong>Even when Packer fix doesn't do anything</strong> to the template,
the template will be outputted to standard out. Things such as configuration
key ordering and indentation may be changed. The output format however, is
pretty-printed for human readability.
</div>
The full list of fixes that the fix command performs is visible in the
help output, which can be seen via `packer fix -h`.
@@ -20,5 +20,10 @@ appear out of order, especially with respect to plugins. In this case,
it is important to pay attention to the timestamp of the log messages
to determine order.
In addition to simply enabling the log, you can set `PACKER_LOG_PATH` in order
to force the log to always go to a specific file when logging is enabled.
Note that even when `PACKER_LOG_PATH` is set, `PACKER_LOG` must be set in
order for any logging to be enabled.
If you find a bug with Packer, please include the detailed log by using
a service such as [gist](http://gist.github.com).
@@ -48,7 +48,8 @@ Optional parameters:
* `environment_vars` (array of strings) - An array of key/value pairs
to inject prior to the execute_command. The format should be
`key=value`.
`key=value`. Packer injects some environmental variables by default
into the environment, as well, which are covered in the section below.
* `execute_command` (string) - The command to use to execute the script.
By default this is `chmod +x {{ .Path }}; {{ .Vars }} {{ .Path }}`. The value of this is
@@ -85,3 +86,17 @@ case is being piped in with the value of "packer".
By setting the `execute_command` to this, your script(s) can run with
root privileges without worrying about password prompts.
## Default Environmental Variables
In addition to being able to specify custom environmental variables using
the `environmental_vars` configuration, the provisioner automatically
defines certain commonly useful environmental variables:
* `PACKER_BUILD_NAME` is set to the name of the build that Packer is running.
This is most useful when Packer is making multiple builds and you want to
distinguish them slightly from a common provisioning script.
* `PACKER_BUILDER_TYPE` is the type of the builder that was used to create
the machine that the script is running on. This is useful if you want to
run only certain parts of the script on systems built with certain builders.
+2 -2
View File
@@ -33,8 +33,8 @@ Along with each key, it is noted whether it is required or not.
information on how to define and configure a provisioner, read the
sub-section on [configuring provisioners in templates](/docs/templates/provisioners.html).
* `post-processors` (optional) is an array of that defines the various
post-processing steps to take with the built images. This is an optional
* `post-processors` (optional) is an array of one or more objects that defines the
various post-processing steps to take with the built images. This is an optional
field. If not specified, then no post-processing will be done. For more
information on what post-processors do and how they're defined, read the
sub-section on [configuring post-processors in templates](/docs/templates/post-processors.html).
+1
View File
@@ -11,6 +11,7 @@
<li><h4>Command-Line</h4></li>
<li><a href="/docs/command-line/introduction.html">Introduction</a></li>
<li><a href="/docs/command-line/build.html">Build</a></li>
<li><a href="/docs/command-line/fix.html">Fix</a></li>
<li><a href="/docs/command-line/validate.html">Validate</a></li>
</ul>