Compare commits

..

1 Commits

Author SHA1 Message Date
Matthew Hooker e5f6958803 update ssh library 2018-04-22 19:47:37 -07:00
3297 changed files with 91156 additions and 416043 deletions
-12
View File
@@ -1,12 +0,0 @@
# Golang CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-go/ for more details
version: 2
jobs:
build:
docker:
- image: circleci/golang:1.11
working_directory: /go/src/github.com/hashicorp/packer
steps:
- checkout
- run: make ci
+2 -4
View File
@@ -1,4 +1,2 @@
* text=auto
*.go text eol=lf
*.sh text eol=lf
common/test-fixtures/root/* eol=lf
common/test-fixtures/root/* eol=lf
+40 -100
View File
@@ -1,7 +1,7 @@
# Contributing to Packer
**First:** if you're unsure or afraid of _anything_, just ask or submit the
issue or pull request anyway. You won't be yelled at for giving your best
issue or pull request anyways. You won't be yelled at for giving your best
effort. The worst that can happen is that you'll be politely asked to change
something. We appreciate any sort of contributions, and don't want a wall of
rules to get in the way of that.
@@ -37,7 +37,7 @@ can quickly merge or address your contributions.
2. The issue is verified and categorized by a Packer collaborator.
Categorization is done via tags. For example, bugs are marked as "bugs" and
simple fixes are marked as "good first issue".
easy fixes are marked as "easy".
3. Unless it is critical, the issue is left for a period of time (sometimes many
weeks), giving outside contributors a chance to address the issue.
@@ -48,42 +48,45 @@ can quickly merge or address your contributions.
5. The issue is closed.
## Setting up Go
## Setting up Go to work on Packer
If you have never worked with Go before, you will have to install its
runtime in order to build packer.
1. [Install go](https://golang.org/doc/install#install)
## Setting up Packer for dev
If/when you have go installed you can already `go get` packer and `make` in
order to compile and test Packer. These instructions target
POSIX-like environments (macOS, Linux, Cygwin, etc.) so you may need to
If you have never worked with Go before, you will have to complete the following
steps in order to be able to compile and test Packer. These instructions target
POSIX-like environments (Mac OS X, Linux, Cygwin, etc.) so you may need to
adjust them for Windows or other shells.
The instructions below are for go 1.7. or later.
1. [Download](https://golang.org/dl) and install Go. The instructions below are
for go 1.7. Earlier versions of Go are no longer supported.
1. Download the Packer source (and its dependencies) by running
2. Set and export the `GOPATH` environment variable and update your `PATH`. For
example, you can add the following to your `.bash_profile` (or comparable
shell startup scripts):
```
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
```
3. Download the Packer source (and its dependencies) by running
`go get github.com/hashicorp/packer`. This will download the Packer source to
`$GOPATH/src/github.com/hashicorp/packer`.
2. When working on Packer, first `cd $GOPATH/src/github.com/hashicorp/packer`
4. When working on Packer, first `cd $GOPATH/src/github.com/hashicorp/packer`
so you can run `make` and easily access other files. Run `make help` to get
information about make targets.
3. Make your changes to the Packer source. You can run `make` in
5. Make your changes to the Packer source. You can run `make` in
`$GOPATH/src/github.com/hashicorp/packer` to run tests and build the Packer
binary. Any compilation errors will be shown when the binaries are
rebuilding. If you don't have `make` you can simply run
`go build -o bin/packer .` from the project root.
4. After running building Packer successfully, use
6. After running building Packer successfully, use
`$GOPATH/src/github.com/hashicorp/packer/bin/packer` to build a machine and
verify your changes work. For instance:
`$GOPATH/src/github.com/hashicorp/packer/bin/packer build template.json`.
5. If everything works well and the tests pass, run `go fmt` on your code before
7. If everything works well and the tests pass, run `go fmt` on your code before
submitting a pull-request.
### Opening an Pull Request
@@ -107,10 +110,6 @@ From there, open your fork in your browser to open a new pull-request.
will break if you `git clone` your fork instead of using `go get` on the main
Packer project.
**Note:** See '[Working with
forks](https://help.github.com/articles/working-with-forks/)' for a better way
to use `git push ...`.
### Pull Request Lifecycle
1. You are welcome to submit your pull request for commentary or review before
@@ -118,95 +117,36 @@ to use `git push ...`.
"[WIP]" to indicate this. It's also a good idea to include specific questions
or items you'd like feedback on.
2. Once you believe your pull request is ready to be merged, you can remove any
"[WIP]" prefix from the title and a core team member will review.
1. Once you believe your pull request is ready to be merged, you can remove any
"[WIP]" prefix from the title and a core team member will review.
3. One of Packer's core team members will look over your contribution and
either merge, or provide comments letting you know if there is anything left
to do. We do our best to provide feedback in a timely manner, but it may take
some time for us to respond. We may also have questions that we need answered
about the code, either because something doesn't make sense to us or because
we want to understand your thought process.
1. One of Packer's core team members will look over your contribution and
either provide comments letting you know if there is anything left to do. We
do our best to provide feedback in a timely manner, but it may take some time
for us to respond.
4. If we have requested changes, you can either make those changes or, if you
disagree with the suggested changes, we can have a conversation about our
reasoning and agree on a path forward. This may be a multi-step process. Our
view is that pull requests are a chance to collaborate, and we welcome
conversations about how to do things better. It is the contributor's
responsibility to address any changes requested. While reviewers are happy to
give guidance, it is unsustainable for us to perform the coding work necessary
to get a PR into a mergeable state.
5. Once all outstanding comments and checklist items have been addressed, your
1. Once all outstanding comments and checklist items have been addressed, your
contribution will be merged! Merged PRs will be included in the next
Packer release. The core team takes care of updating the
[CHANGELOG.md](../CHANGELOG.md) as they merge.
6. In rare cases, we might decide that a PR should be closed without merging.
We'll make sure to provide clear reasoning when this happens.
Packer release. The core team takes care of updating the CHANGELOG as they
merge.
1. In rare cases, we might decide that a PR should be closed. We'll make sure to
provide clear reasoning when this happens.
### Tips for Working on Packer
#### Getting Your Pull Requests Merged Faster
It is much easier to review pull requests that are:
1. Well-documented: Try to explain in the pull request comments what your
change does, why you have made the change, and provide instructions for how
to produce the new behavior introduced in the pull request. If you can,
provide screen captures or terminal output to show what the changes look
like. This helps the reviewers understand and test the change.
2. Small: Try to only make one change per pull request. If you found two bugs
and want to fix them both, that's _awesome_, but it's still best to submit
the fixes as separate pull requests. This makes it much easier for reviewers
to keep in their heads all of the implications of individual code changes,
and that means the PR takes less effort and energy to merge. In general, the
smaller the pull request, the sooner reviewers will be able to make time to
review it.
3. Passing Tests: Based on how much time we have, we may not review pull
requests which aren't passing our tests. (Look below for advice on how to
run unit tests). If you need help figuring out why tests are failing, please
feel free to ask, but while we're happy to give guidance it is generally
your responsibility to make sure that tests are passing. If your pull request
changes an interface or invalidates an assumption that causes a bunch of
tests to fail, then you need to fix those tests before we can merge your PR.
If we request changes, try to make those changes in a timely manner. Otherwise,
PRs can go stale and be a lot more work for all of us to merge in the future.
Even with everyone making their best effort to be responsive, it can be
time-consuming to get a PR merged. It can be frustrating to deal with
the back-and-forth as we make sure that we understand the changes fully. Please
bear with us, and please know that we appreciate the time and energy you put
into the project.
#### Working on forks
The easiest way to work on a fork is to set it as a remote of the Packer
project. After following the steps in "Setting up Go to work on Packer":
1. Navigate to the code:
`cd $GOPATH/src/github.com/hashicorp/packer`
2. Add the remote by running:
`git remote add <name of remote> <github url of fork>`
For example:
`git remote add mwhooker https://github.com/mwhooker/packer.git`
3. Checkout a feature branch:
`git checkout -b new-feature`
4. Make changes.
1. Navigate to `$GOPATH/src/github.com/hashicorp/packer`
2. Add the remote by running
`git remote add <name of remote> <github url of fork>`. For example:
`git remote add mwhooker https://github.com/mwhooker/packer.git`.
3. Checkout a feature branch: `git checkout -b new-feature`
4. Make changes
5. (Optional) Push your changes to the fork:
`git push -u <name of remote> new-feature`
This way you can push to your fork to create a PR, but the code on disk still
@@ -243,7 +183,7 @@ Packer has [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing)
for various builders. These typically require an API key (AWS, GCE), or
additional software to be installed on your computer (VirtualBox, VMware).
If you're working on a new builder or builder feature and want to verify it is
If you're working on a new builder or builder feature and want verify it is
functioning (and also hasn't broken anything else), we recommend running the
acceptance tests.
+1 -1
View File
@@ -2,7 +2,7 @@ _Please read these instructions before submitting_
**DELETE THIS TEMPLATE BEFORE SUBMITTING**
_Only use GitHub issues to report bugs or feature requests, see
_Only use Github issues to report bugs or feature requests, see
https://www.packer.io/community.html_
For example, _Timeouts waiting for SSH/WinRM_ are generally not bugs within packer and are better addressed by the mailing list. Ask on the mailing list if you are unsure.
-6
View File
@@ -1,11 +1,5 @@
**DELETE THIS TEMPLATE BEFORE SUBMITTING**
In order to have a good experience with our community, we recommend that you
read the contributing guidelines for making a PR, and understand the lifecycle
of a Packer PR:
https://github.com/hashicorp/packer/blob/master/.github/CONTRIBUTING.md#opening-an-pull-request
Describe the change you are making here!
Please include tests. Check out these examples:
+1 -3
View File
@@ -4,7 +4,6 @@
/src
/website/.sass-cache
/website/build
/website/tmp
.DS_Store
.vagrant
.idea
@@ -24,5 +23,4 @@ packer-test*.log
.idea/
*.iml
Thumbs.db
/packer.exe
.project
/packer.exe
+7 -18
View File
@@ -1,35 +1,24 @@
env:
- USER=travis
os:
- windows
- linux
- osx
- USER=travis
sudo: false
language: go
go:
- 1.11.x
- master
- 1.8.x
- 1.9.x
- 1.x
before_install:
- >
if [[ "$TRAVIS_OS_NAME" == "windows" ]];
then choco install -y make;
fi
install:
- make deps
script:
- df -h
- make ci
- GOMAXPROCS=2 make ci
branches:
only:
- master
matrix:
allow_failures:
- go: master
- os: windows
fast_finish: true
+5 -493
View File
@@ -1,504 +1,16 @@
## 1.3.4 (upcoming)
### IMPROVEMENTS:
* builder/alicloud: delete copied image and snapshots if corresponding options
are specified [GH-7050]
* builder/amazon: allow to interpolate more variables [GH-7059]
* builder/amazon: Check that the KMS key ID is valid [GH-7090]
* builder/amazon: Clean up logging for aws waiters so that it only runs once
per builder [GH-7080]
* builder/amazon: don't Cleanup Temp Keys when there is no communicator to
avoid a panic [GH-7100] [GH-7095]
* builder/amazon: Don't try to guess region from metadata if not set & update
aws-sdk-go [GH-7230]
* builder/amazon: Import error messages should now contain reason for failure
[GH-7207]
* builder/azure: add certificate authentication [GH-7189]
* builder/azure: allow to configure disk caching [GH-7061]
* builder/azure: use deallocate instead of just power-off [GH-7203]
* builder/hyperv: Add support for legacy network adapters on Hyper-V. [GH-7128]
* builder/hyperv: Allow user to set `version` option in the New-VM command.
[GH-7136]
* builder/openstack: Add `volume_size` option [GH-7130]
* builder/openstack: Don't require network v2 [GH-6933]
* builder/openstack: Support for tagging new images [GH-7037]
* builder/qemu: Add configuration options to specify cpu count and memory size
[GH-7156]
* builder/qemu: Add support for whpx accelerator to qemu builder [GH-7151]
* builder/vmware: Escape query as suggested in issue #7200 [GH-7223]
* core/shell: Add env vars "PACKER_HTTP_IP" and "PACKER_HTTP_PORT" to shell
provisioners [GH-7075]
* core: allow to use `-except` on post-processors [GH-7183]
* core: Clean up internal handling and creation of temporary directories
[GH-7102]
* core: Deprecate mitchellh/go-homedir package in favor of os/user [GH-7062]
* core: Download checksum match failures will now log the received checksum.
[GH-7210]
* core: Explicitly set ProxyFromEnvironment in httpclients when creating an aws
session [GH-7226]
* core: make packer inspect not print sensitive variables [GH-7084]
* post-processor/google: Add new `guest-os-features` option. [GH-7218]
* postprocessor/docker-import: Added `change` support [GH-7127]
* provisioner/ansible-remote: add `-o IdentitiesOnly=yes`as a default flag
[GH-7115]
* provisioner/chef-client: Elevated support for chef-client provisioner
[GH-7078]
* provisioner/puppet: Elevated support for puppet-* provisioner [GH-7078]
* provisioner/windows-restart: wait for already-scheduled reboot [GH-7056] and
ignore reboot specific errors [GH-7071]
### BUG FIXES:
* builder/azure: Ensure the Windows Guest Agent is fully functional before
Sysprep is executed. [GH-7176]
* builder/azure: Fix snapshot regression [GH-7111]
* builder/docker: Ensure that entrypoint and arguments get passed to docker,
not the image. [GH-7091]
* builder/hcloud: fix go mod dependency [GH-7099]
* builder/hcloud: prevent panic when ssh key was not passed [GH-7118]
* builder/hyperv: Fix the Hyper-V gen 1 guest boot order. [GH-7147]
* builder/hyperv: hyper-v builder no longer ignores `ssh_host` option.
[GH-7154]
* builder/oracle-oci: Fix crash that occurs when image is nil [GH-7126]
* builder/parallels: Fix attaching prl tools [GH-7158]
* builder/virtualbox: Fix handling of portcount argument for version 6 beta
[GH-7174] [GH-7094]
* builder/vmware: Fix bug caused by 'nil' dir field in artifact struct when
building locally [GH-7116]
* communicator/docker: Fix docker file provisioner on Windows [GH-7163]
* core: prioritize AppData over default user directory ( UserProfile )
[GH-7166]
* core: removed a flaky race condition in tests [GH-7119]
* postprocessor/vsphere: Stop setting HDDOrder, since it was breaking uploads
[GH-7108]
## 1.3.3 (December 5, 2018)
### IMPROVEMENTS:
* builder/alicloud: Add options for system disk properties [GH-6939]
* builder/alicloud: Apply tags to relevant snapshots [GH-7040]
* builder/alicloud: Support creating image without data disks [GH-7022]
* builder/amazon: Add option for skipping TLS verification [GH-6842]
* builder/azure: Add options for Managed Image OS Disk and Data Disk snapshots
[GH-6980]
* builder/hcloud: Add `snapshot_labels` option to hcloud builder [GH-7046]
* builder/hcloud: Add ssh_keys config to hcloud builder [GH-7028]
* builder/hcloud: Update hcloud-go version and support builds using rescue mode
[GH-7034]
* builder/oracle: Parameterized volume size support for Oracle classic builder
[GH-6918]
* builder/parallels: Add configuration options to parallels builder to specify
cpu count and memory size [GH-7018]
* builder/virtualbox: Add configuration options to virtualbox builder to
specify cpu count and memory size [GH-7017]
* builder/virtualbox: expose the VBoxManage export --iso option [GH-5950]
* builder/vmware: Add configuration options to vmware builder to specify cpu
count and memory size [GH-7019]
* builder/vmware: Add new display_name template option [GH-6984]
* builder/vmware: Extend vmware-vmx builder to allow esxi builds. [GH-4591]
[GH-6927]
* builder/vmware: Validate username/password for ovftool during prepare.
[GH-6977]
* builder/vmware: Warn users if their vmx_data overrides data that Packer uses
the template engine to set in its default vmx template. [GH-6987]
* communicator/ssh: Expand user path for SSH private key [GH-6946]
* core: Add a sed template engine [GH-6580]
* core: More explicit error message in rpc/ui.go [GH-6981]
* core: Replaced unsafe method of determining homedir with os/user
implementation [GH-7036]
* core: Update vagrantfile's go version. [GH-6841]
* post-processor/amazon-import: Support ova, raw, vmdk, and vhdx formats in the
amazon-import post-processor. [GH-6938]
* post-processor/vsphere-template: Add option to snapshot vm before marking as
template [GH-6969]
* provisioner/breakpoint: Add a new breakpoint provisioner. [GH-7058]
* provisioner/powershell: Allow Powershell provisioner to use service accounts
[GH-6972]
* provisioner/shell: Add PauseAfter option to shell provisioner [GH-6913]
### BUG FIXES:
* builder/amazon: Better error handling of region/credential guessing from
metadata [GH-6931]
* builder/amazon: move region validation to run so that we don't break
validation when no credentials are set [GH-7032]
* builder/hyperv: Remove -Copy:$false when calling Hyper-V\Compare-VM
compatability report [GH-7030]
* builder/qemu: Do not set detect-zeroes option when we want it "off" [GH-7064]
* builder/vmware-esxi: Create export directories for vmx and ovf file types
[GH-6985]
* builder/vmware: Correctly parse version for VMware Fusion Tech Preview
[GH-7016]
* builder/vmware: Escape vSphere username when putting it into the export call
[GH-6962]
* post-processor/vagrant: Add "hvf" as a libvirt driver [GH-6955]
* provisioner/ansible: inventory is no longer set to inventory_directory
[GH-7065]
## 1.3.2 (October 29, 2018)
### IMPROVEMENTS:
* builder/alicloud: Add new `disable_stop_instance` option. [GH-6764]
* builder/alicloud: Support adding tags to image. [GH-6719]
* builder/alicloud: Support ssh with private ip address. [GH-6688]
* builder/amazon: Add support to explicitly control ENA support [GH-6872]
* builder/amazon: Add suppport for `vpc_filter`, `subnet_filter`, and
`security_group_filter`. [GH-6374]
* builder/amazon: Add validation for required `device_name` parameter in
`block_device_mappings`. [GH-6845]
* builder/amazon: Clean up security group wait code. [GH-6843]
* builder/amazon: Update aws-sdk-go to v1.15.54, adding support for
`credential_source`. [GH-6849]
* builder/amazon: Use DescribeRegions for aws region validation. [GH-6512],
[GH-6904]
* builder/azure: Add new `shared_image_gallery` option. [GH-6798]
* builder/googlecompute: Return an error if `startup_script_file` is specified,
but file does not exist. [GH-6848]
* builder/hcloud: Add Hetzner Cloud builder. [GH-6871]
* builder/openstack: Add new `disk_format` option. [GH-6702]
* builder/openstack: Fix bug where `source_image_name` wasn't being used to
properly find a UUID. [GH-6751]
* builder/openstack: Wait for volume availability when cleaning up [GH-6703]
* builder/qemu: Add `disk_detect_zeroes` option. [GH-6827]
* builder/scaleway: Add `boottype` parameter to config. [GH-6772]
* builder/scaleway: Update scaleway-cli vendor. [GH-6771]
* core: New option to add timestamps to UI output. [GH-6784]
* post-processor/vagrant-cloud: Validate vagrant cloud auth token doing an auth
request [GH-6914]
* provisioner/file: Improve error messaging when file destination is a
directory with no trailing slash. [GH-6756]
* provisioner/powershell: Provide better error when Packer can't find
Powershell executable. [GH-6817]
* provisioner/shell-local: Add ability to specify OSs where shell-local can run
[GH-6878]
### BUG FIXES:
* builder/alicloud: Fix ssh configuration pointer issues that could cause a bug
[GH-6720]
* builder/alicloud: Fix type error in step_create_tags [GH-6763]
* builder/amazon: Error validating credentials is no longer obscured by a
region validation error. and some region validation refactors and
improvements [GH-6865]
* builder/amazon: Fix error calculating defaults in AWS waiters. [GH-6727]
* builder/amazon: Increase default wait for image import to one hour. [GH-6818]
* builder/amazon: Waiter now fails rather than hanging for extra time when an
image import fails. [GH-6747]
* builder/azure: Updated Azure/go-ntlmssp dependency to resolve an issue with
the winrm communicator not connecting to Windows machines requiring NTLMv2
session security
* builder/digitalocean: Fix ssh configuration pointer issues that could cause a
panic [GH-6729]
* builder/hyperv/vmcx: Allow to set generation from buildfile [GH-6909]
* builder/scaleway: Fix issues with ssh keys. [GH-6768]
* core: Fix error where logging was always enabled when Packer was run from
inside Terraform. [GH-6758]
* core: Fix issue with with names containing spaces in ESX5Driver and in ssh
communicator [GH-6891], [GH-6823]
* core: Fix logger so it doesn't accidentally try to format unescaped strings.
[GH-6824]
* core: Fix race conditions in progress bar code [GH-6858], [GH-6788],
[GH-6851]
* core: Fix various places in multiple builders where config was not being
passed as a pointer. [GH-6739]
* post-processor/manifest: No longer provides an empty ID string for Azure's
managed image artifact [GH-6822]
* provisioner/powershell: Fix a bug in the way we set the ProgressPreference
variable in the default `execute_command` [GH-6838]
* provisioner/windows-restart: Fix extraneous break which forced early exit
from our wait loop. [GH-6792]
## 1.3.1 (September 13, 2018)
### IMPROVEMENTS:
* builder/amazon: automatically decode encoded authorization messages if
possible [GH-5415]
* builder:amazon: Optional cleanup of the authorized keys file [GH-6713]
* builder/qemu: Fixed bug where a -device in the qemuargs would override the default network settings, resulting in no network [GH-6807]
### BUG FIXES:
* builder/amazon: fix bugs relating to spot instances provisioning [GH-6697]
[GH-6693]
* builder/openstack: fix ssh keypair not attached [GH-6701]
* core: progressbar: fix deadlock locking builds afer first display [GH-6698]
## 1.3.0 (September 11, 2018)
### IMPROVEMENTS:
* azure/arm: Retry cleanup of individual resources on error [GH-6644]
* builder/alicloud: Support source image coming from marketplace [GH-6588]
* builder/amazon-chroot: Add new `root_volume_type` option. [GH-6669]
* builder/amazon-chroot: If you have a PV source AMI, with the Amazon Chroot
builder, and the destination AMI is type HVM, you can now enable
ena_support, example: [GH-6670]
* builder/amazon-chroot: New feature `root_volume_tags` to tag the created
volumes. [GH-6504]
* builder/amazon: Create a random interim AMI name when encrypt_boot is true so
that ami name is not searchable. [GH-6657]
* builder/azure: Implement clean_image_name template engine. [GH-6558]
* builder/cloudstack: Add option to use a fixed port via public_port. [GH-6532]
* builder/digitalocean: Add support for tagging to instances [GH-6546]
* builder/googlecompute: Add new `min_cpu_platform` feature [GH-6607]
* builder/googlecompute: Update the list of public image projects that we
search, based on GCE documentation. [GH-6648]
* builder/lxc: Allow unplivileged LXC containers. [GH-6279]
* builder/oci: Add `metadata` feature to Packer config. [GH-6498]
* builder/openstack: Add support for getting config from clouds-public.yaml.
[GH-6595]
* builder/openstack: Add support for ports. [GH-6570]
* builder/openstack: Add support for source_image_filter. [GH-6490]
* builder/openstack: Migrate floating IP usage to Network v2 API from Compute
API. [GH-6373]
* builder/openstack: Support Block Storage volumes as boot volume. [GH-6596]
* builder/oracle-oci: Add support for freeform tagging of OCI images [GH-6338]
* builder/qemu: add ssh agent support. [GH-6541]
* builder/qemu: New `use_backing_file` feature [GH-6249]
* builder/vmware-iso: Add support for disk compaction [GH-6411]
* builder/vmware-iso: Try to use ISO files uploaded to the datastore when
building remotely instead of uploading them freshly every time [GH-5165]
* command/validate: Warn users if config needs fixing. [GH-6423]
* core: Add a 'split' function to parse template variables. [GH-6357]
* core: Add a template function allowing users to read keys from consul
[GH-6577]
* core: Add a template function allowing users to read keys from vault
[GH-6533]
* core: Add progress-bar to download step. [GH-5851]
* core: Create a new root-level Packer template option, "sensitive-variables"
which allows users to list which variables they would like to have scrubbed
from the Packer logs. [GH-6610]
* core: Create new config options, "boot_keygroup_interval" and
"boot_key_interval" that can be set at the builder-level to supercede
PACKER_KEY_INTERVAL for the bootcommand. [GH-6616]
* core: Deduplicate ui and log lines that stream to terminal [GH-6611]
* core: Refactor and deduplicate ssh code across builders. This should be a no-
op but is a big win for maintainability. [GH-6621] [GH-6613]
* post-processor/compress: Add support for xz compression [GH-6534]
* post-processor/vagrant: Support for Docker images. [GH-6494]
* post-processor/vsphere: Add new `esxi_host` option. [GH-5366]
* postprocessor/vagrant: Add support for Azure. [GH-6576]
* provisioner/ansible: Add new "extra var", packer_http_addr. [GH-6501]
* provisioner/ansible: Enable {{.WinRMPassword}} template engine. [GH-6450]
* provisioner/shell-local: Create PACKER_HTTP_ADDR environment variable
[GH-6503]
### BUG FIXES:
* builder/amazon-ebssurrogate: Clean up volumes at end of build. [GH-6514]
* builder/amazon: Increase default waiter timeout for AWS
WaitUntilImageAvailable command [GH-6601]
* builder/amazon: Increase the MaxRetries in the Amazon client from the default
to 20, to work around users who regularly reach their requestlimit and are
being throttled. [GH-6641]
* builder/amazon: Properly apply environment overrides to our custom-written
waiters. [GH-6649]
* builder/azure: Generated password satisfies Azure password requirements
[GH-6480]
* builder/hyper-v: Buider no longer errors if skip_compaction isn't true when
skip_export is true, and compaction efficiency is improved [GH-6393]
* builder/lxc: Correctly pass "config" option to "lxc launch". [GH-6563]
* builder/lxc: Determine lxc root according to the running user [GH-6543]
* builder/lxc: Fix file copying for unprivileged LXC containers [GH-6544]
* builder/oracle-oci: Update OCI sdk, fixing validation bug that occurred when
RSA key was encrypted. [GH-6492]
* builder/vmware-iso: Fix crash caused by invalid datacenter url. [GH-6529]
* builder/vmware: Maintain original boot order during CreateVMX step for
vmware-iso builder [GH-6204]
* communicator/chroot: Fix quote escaping so that ansible provisioner works
properly. [GH-6635]
* core: Better error handling in downloader when connection error occurs.
[GH-6557]
* core: Fix broken pathing checks in checksum files. [GH-6525]
* provisioner/shell Create new template option allowing users to choose to
source env vars from a file rather than declaring them inline. This
resolves a bug that occurred when users had complex quoting in their
`execute_command`s [GH-6636]
* provisioner/shell-local: Windows inline scripts now default to being appended
with ".cmd", fixing a backwards incompatibility in v1.2.5 [GH-6626]
* provisioner/windows-restart: Provisioner now works when used in conjuction
with SSH communicator [GH-6606]
### BACKWARDS INCOMPATIBILITIES:
* builder/amazon: "owners" field on source_ami_filter is now required for
secuirty reasons. [GH-6585]
* builder/vmware-iso: validation will fail for templates using esxi that have the "disk_type_id" set to something other than "thin" or "" and that do not have "skip_compaction": true also set. Use `packer fix` to fix this. [GH-6411]
## 1.2.5 (July 16, 2018)
### BUG FIXES:
* builder/alickoud: Fix issue where internet_max_bandwidth_out template option
was not being passed to builder. [GH-6416]
* builder/alicloud: Fix an issue with VPC cleanup. [GH-6418]
* builder/amazon-chroot: Fix communicator bug that broke chroot builds.
[GH-6363]
* builder/amazon: Replace packer's waiters with those from the AWS sdk, solving
several timeout bugs. [GH-6332]
* builder/azure: update azure-sdk-for-go, fixing 32-bit build errors. [GH-6479]
* builder/azure: update the max length of managed_image_resource_group to match
new increased length of 90 characters. [GH-6477]
* builder/hyper-v: Fix secure boot template feature so that it properly passes
the temolate for MicrosoftUEFICertificateAuthority. [GH-6415]
* builder/hyperv: Fix bug in HyperV IP lookups that was causing breakages in
FreeBSD/OpenBSD builds. [GH-6416]
* builder/qemu: Fix error race condition in qemu builder that caused convert to
fail on ubuntu 18.x [GH-6437]
* builder/qemu: vnc_bind_address was not being passed to qemu. [GH-6467]
* builder/virtualbox: Allow iso_url to be a symlink. [GH-6370]
* builder/vmware: Don't fail on DHCP lease files that cannot be read, fixing
bug where builder failed on NAT networks that don't serve DHCP. [GH-6415]
* builder/vmware: Fix bug where we couldn't discover IP if vm_name differed
from the vmx displayName. [GH-6448]
* builder/vmware: Fix validation to prevent hang when remopte_password is not
sent but vmware is building on esxi. [GH-6424]
* builder/vmware:Correctly default the vm export format to ovf; this is what
the docs claimed we already did, but we didn't. [GH-4538]
* communicator/winrm: Revert an attempt to determine whether remote upload
destinations were files or directories, as this broke uploads on systems
without Powershell installed. [GH-6481]
* core: Fix bug in parsing of iso checksum files that arose when setting
iso_url to a relative filepath. [GH-6488]
* core: Fix Packer crash caused by improper error handling in the downloader.
[GH-6381]
* fix: Fix bug where fixer for ssh_private_ip that failed when boolean values
are passed as strings. [GH-6458]
* provisioner/powershell: Make upload of powershell variables retryable, in
case of system restarts. [GH-6388]
### IMPROVEMENTS:
* builder/amazon: Add the ap-northeast-3 region. [GH-6385]
* builder/amazon: Spot requests may now have tags applied using the `spot_tags`
option [GH-5452]
* builder/cloudstack: Add support for Projectid and new config option
prevent_firewall_changes. [GH-6487]
* builder/openstack: Add support for token authorization and cloud.yaml.
[GH-6362]
* builder/oracle-oci: Add new "instance_name" template option. [GH-6408]
* builder/scaleway: Add new "bootscript" parameter, allowing the user to not
use the default local bootscript [GH-6439]
* builder/vmware: Add support for linked clones to vmware-vmx. [GH-6394]
* debug: The -debug flag will now cause Packer to pause between provisioner
scripts in addition to Packer steps. [GH-4663]
* post-processor/googlecompute-import: Added new googlecompute-import post-
processor [GH-6451]
* provisioner/ansible: Add new "playbook_files" option to execute multiple
playbooks within one provisioner call. [GH-5086]
## 1.2.4 (May 29, 2018)
## (UNRELEASED)
### BUG FIXES:
* builder/amazon: Can now force the chroot builder to mount an entire block
device instead of a partition [GH-6194]
* builder/azure: windows-sql-cloud is now in the default list of projects to
check for provided images. [GH-6210]
* builder/chroot: A new template option, `nvme_device_path` has been added to
provide a workaround for users who need the amazon-chroot builder to mount
a NVMe volume on their instances. [GH-6295]
* builder/hyper-v: Fix command for mounting multiple disks [GH-6267]
* builder/hyperv: Enable IP retrieval for Server 2008 R2 hosts. [GH-6219]
* builder/hyperv: Fix bug in MAC address specification on Hyper-V. [GH-6187]
* builder/parallels-pvm: Add missing disk compaction step. [GH-6202]
* builder/vmware-esxi: Remove floppy files from the remote server on cleanup.
[GH-6206]
* communicator/winrm: Updated dependencies to fix a race condition [GH-6261]
* core: When using `-on-error=[abort|ask]`, output the error to the user.
[GH-6252]
* provisioner/puppet: Extra-Arguments are no longer prematurely
interpolated.[GH-6215]
* provisioner/shell: Remove file stat that was causing problems uploading files
[GH-6239]
### IMPROVEMENTS:
* builder/amazon: Amazon builders other than `chroot` now support T2 unlimited
instances [GH-6265]
* builder/azure: Allow device login for US government cloud. [GH-6105]
* builder/azure: Devicelogin Support for Windows [GH-6285]
* builder/azure: Enable simultaneous builds within one resource group.
[GH-6231]
* builder/azure: Faster deletion of Azure Resource Groups. [GH-6269]
* builder/azure: Updated Azure SDK to v15.0.0 [GH-6224]
* builder/hyper-v: Hyper-V builds now connect to vnc display by default when
building [GH-6243]
* builder/hyper-v: New `use_fixed_vhd_format` allows vm export in an Azure-
compatible format [GH-6101]
* builder/hyperv: New config option for specifying what secure boot template to
use, allowing secure boot of linux vms. [GH-5883]
* builder/qemu: Add support for hvf accelerator. [GH-6193]
* builder/scaleway: Fix SSH communicator connection issue. [GH-6238]
* core: Add opt-in Packer top-level command autocomplete [GH-5454]
* post-processor/shell-local: New options have been added to create feature
parity with the shell-local provisioner. This feature now works on Windows
hosts. [GH-5956]
* provisioner/chef: New config option allows user to skip cleanup of chef
client staging directory. [GH-4300]
* provisioner/shell-local: Can now access automatically-generated WinRM
password as variable [GH-6251]
* provisoner/shell-local: New options have been added to create feature parity
with the shell-local post-processor. This feature now works on Windows
hosts. [GH-5956]
* builder/virtualbox: Use HTTPS to download guest editions, now that it's
available. [GH-6406]
## 1.2.3 (April 25, 2018)
### BUG FIXES:
* builder/azure: Azure CLI may now be logged into several accounts. [GH-6087]
* builder/ebssurrogate: Snapshot all launch devices. [GH-6056]
* builder/hyper-v: Fix CopyExportedVirtualMachine script so it works with
links. [GH-6082]
* builder/oracle-classic: Fix panics when cleaning up resources that haven't
been created. [GH-6095]
* builder/parallels: Allow user to cancel build while the OS is starting up.
[GH-6166]
* builder/qemu: Avoid warning when using raw format. [GH-6080]
* post-processor/vagrant: Large VMDKs should no longer show a 0-byte size on OS X. [GH-6084]
* builder/scaleway: Fix compilation issues on solaris/amd64. [GH-6069]
* builder/virtualbox: Fix broken scancodes in boot_command. [GH-6067]
* builder/vmware-iso: Fail in validation if user gives wrong remote_type value.
[GH-4563]
* builder/vmware: Fixed a case-sensitivity issue when determing the network
type during the cloning step in the vmware-vmx builder. [GH-6057]
* builder/vmware: Fixes the DHCP lease and configuration pathfinders for VMware
Player. [GH-6096]
* builder/vmware: Multi-disk VM's can be properly handled by the compacting
stage. [GH-6074]
* common/bootcommand: Fix numerous bugs in the boot command code, and make
supported features consistent across builders. [GH-6129]
* communicator/ssh: Stop trying to discover whether destination is a directory
from uploader. [GH-6124]
* post-processor/vagrant: Large VMDKs should no longer show a 0-byte size on OS
X. [GH-6084]
* post-processor/vsphere: Fix encoding of spaces in passwords for upload.
[GH-6110]
* provisioner/ansible: Pass the inventory_directory configuration option to
ansible -i when it is set. [GH-6065]
* provisioner/powershell: fix bug with SSH communicator + cygwin. [GH-6160]
* provisioner/powershell: The {{.WinRMPassword}} template variable now works
with parallel builders. [GH-6144]
* common/bootcommand: Fix numerous bugs in the boot command code, and make supported features consistent across builders. [GH-6129]
### IMPROVEMENTS:
* builder/alicloud: Update aliyungo common package. [GH-6157]
* builder/amazon: Expose more source ami data as template variables. [GH-6088]
* builder/amazon: Setting `force_delete` will only delete AMIs owned by the
user. This should prevent failures where we try to delete an AMI with a
matching name, but owned by someone else. [GH-6111]
* builder/azure: Users of Powershell provisioner may access the randomly-
generated winrm password using the template variable {{.WinRMPassword}}.
[GH-6113]
* builder/google: Users of Powershell provisioner may access the randomly-
generated winrm password using the template variable {{.WinRMPassword}}.
[GH-6141]
* builder/hyper-v: User can now configure hyper-v disk block size. [GH-5941]
* builder/openstack: Add configuration option for `instance_name`. [GH-6041]
* builder/oracle-classic: Better validation of destination image name.
[GH-6089]
* builder/oracle-oci: New config options for user data and user data file.
[GH-6079]
* builder/oracle-oci: use the official OCI sdk instead of handcrafted client.
[GH-6142]
* builder/triton: Add support to Skip TLS Verification of Triton Certificate.
[GH-6039]
* provisioner/ansible: Ansible users may provide a custom inventory file.
[GH-6107]
* provisioner/file: New `generated` tag allows users to upload files created
during Packer run. [GH-3891]
user. This should prevent failures where we try to delete an AMI with
a matching name, but owned by someone else. [GH-6111]
## 1.2.2 (March 26, 2018)
+2 -3
View File
@@ -2,7 +2,7 @@
# builders
/builder/alicloud/ @chhaj5236
/builder/alicloud/ dongxiao.zzh@alibaba-inc.com
/builder/amazon/ebssurrogate/ @jen20
/builder/amazon/ebsvolume/ @jen20
/builder/azure/ @boumenot
@@ -14,8 +14,7 @@
/builder/profitbricks/ @jasmingacic
/builder/triton/ @jen20 @sean-
/builder/ncloud/ @YuSungDuk
/builder/scaleway/ @sieben @mvaude @jqueuniet @fflorens @brmzkw
/builder/hcloud/ @LKaemmerling
/builder/scaleway/ @dimtion @edouardb
# provisioners
-30
View File
@@ -1,30 +0,0 @@
FROM ubuntu:16.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && apt-get install -y \
locales \
openssh-server \
sudo
RUN locale-gen en_US.UTF-8
RUN if ! getent passwd vagrant; then useradd -d /home/vagrant -m -s /bin/bash vagrant; fi \
&& echo 'vagrant ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers \
&& mkdir -p /etc/sudoers.d \
&& echo 'vagrant ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/vagrant \
&& chmod 0440 /etc/sudoers.d/vagrant
RUN mkdir -p /home/vagrant/.ssh \
&& chmod 0700 /home/vagrant/.ssh \
&& wget --no-check-certificate \
https://raw.github.com/hashicorp/vagrant/master/keys/vagrant.pub \
-O /home/vagrant/.ssh/authorized_keys \
&& chmod 0600 /home/vagrant/.ssh/authorized_keys \
&& chown -R vagrant /home/vagrant/.ssh
RUN mkdir -p /run/sshd
CMD /usr/sbin/sshd -D \
-o UseDNS=no \
-o PidFile=/tmp/sshd.pid
+15 -38
View File
@@ -4,15 +4,11 @@ VET?=$(shell ls -d */ | grep -v vendor | grep -v website)
GITSHA:=$(shell git rev-parse HEAD)
# Get the current local branch name from git (if we can, this may be blank)
GITBRANCH:=$(shell git symbolic-ref --short HEAD 2>/dev/null)
GOFMT_FILES?=$$(find . -not -path "./vendor/*" -name "*.go")
GOOS=$(shell go env GOOS)
GOARCH=$(shell go env GOARCH)
GOPATH=$(shell go env GOPATH)
# gofmt
UNFORMATTED_FILES=$(shell find . -not -path "./vendor/*" -name "*.go" | xargs gofmt -s -l)
EXECUTABLE_FILES=$(shell find . -type f -executable | egrep -v '^\./(website/[vendor|tmp]|vendor/|\.git|bin/|scripts/|pkg/)' | egrep -v '.*(\.sh|\.bats|\.git)' | egrep -v './provisioner/ansible/test-fixtures/exit1')
# Get the git commit
GIT_DIRTY=$(shell test -n "`git status --porcelain`" && echo "+CHANGES" || true)
GIT_COMMIT=$(shell git rev-parse --short HEAD)
@@ -21,11 +17,9 @@ GOLDFLAGS=-X $(GIT_IMPORT).GitCommit=$(GIT_COMMIT)$(GIT_DIRTY)
export GOLDFLAGS
.PHONY: bin checkversion ci default deps fmt fmt-docs fmt-examples generate releasebin test testacc testrace updatedeps
default: deps generate test dev
default: deps generate testrace dev releasebin package dev fmt fmt-check mode-check fmt-docs fmt-examples
ci: testrace
ci: deps test
release: deps test releasebin package ## Build a release build
@@ -47,10 +41,10 @@ package:
@sh -c "$(CURDIR)/scripts/dist.sh $(VERSION)"
deps:
@go get golang.org/x/tools/cmd/goimports
@go get golang.org/x/tools/cmd/stringer
@go get -u github.com/mna/pigeon
@go get github.com/kardianos/govendor
@govendor sync
dev: deps ## Build and install a development build
@grep 'const VersionPrerelease = ""' version/version.go > /dev/null ; if [ $$? -eq 0 ]; then \
@@ -64,28 +58,11 @@ dev: deps ## Build and install a development build
@cp $(GOPATH)/bin/packer pkg/$(GOOS)_$(GOARCH)
fmt: ## Format Go code
@gofmt -w -s main.go $(UNFORMATTED_FILES)
@gofmt -w -s $(GOFMT_FILES)
fmt-check: ## Check go code formatting
@echo "==> Checking that code complies with gofmt requirements..."
@if [ ! -z "$(UNFORMATTED_FILES)" ]; then \
echo "gofmt needs to be run on the following files:"; \
echo "$(UNFORMATTED_FILES)" | xargs -n1; \
echo "You can use the command: \`make fmt\` to reformat code."; \
exit 1; \
else \
echo "Check passed."; \
fi
$(CURDIR)/scripts/gofmtcheck.sh $(GOFMT_FILES)
mode-check: ## Check that only certain files are executable
@echo "==> Checking that only certain files are executable..."
@if [ ! -z "$(EXECUTABLE_FILES)" ]; then \
echo "These files should not be executable or they must be white listed in the Makefile:"; \
echo "$(EXECUTABLE_FILES)" | xargs -n1; \
exit 1; \
else \
echo "Check passed."; \
fi
fmt-docs:
@find ./website/source/docs -name "*.md" -exec pandoc --wrap auto --columns 79 --atx-headers -s -f "markdown_github+yaml_metadata_block" -t "markdown_github+yaml_metadata_block" {} -o {} \;
@@ -101,25 +78,25 @@ generate: deps ## Generate dynamically generated code
goimports -w common/bootcommand/boot_command.go
gofmt -w command/plugin.go
test: fmt-check mode-check vet ## Run unit tests
test: deps fmt-check ## Run unit tests
@go test $(TEST) $(TESTARGS) -timeout=2m
@go tool vet $(VET) ; if [ $$? -eq 1 ]; then \
echo "ERROR: Vet found problems in the code."; \
exit 1; \
fi
# testacc runs acceptance tests
testacc: deps generate ## Run acceptance tests
@echo "WARN: Acceptance tests will take a long time to run and may cost money. Ctrl-C if you want to cancel."
PACKER_ACC=1 go test -v $(TEST) $(TESTARGS) -timeout=45m
testrace: fmt-check mode-check vet ## Test with race detection enabled
@go test -race $(TEST) $(TESTARGS) -timeout=2m -p=8
testrace: deps ## Test for race conditions
@go test -race $(TEST) $(TESTARGS) -timeout=2m
updatedeps:
@echo "INFO: Packer deps are managed by govendor. See .github/CONTRIBUTING.md"
vet: ## Vet Go code
@go tool vet $(VET) ; if [ $$? -eq 1 ]; then \
echo "ERROR: Vet found problems in the code."; \
exit 1; \
fi
help:
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.PHONY: bin checkversion ci default deps fmt fmt-docs fmt-examples generate releasebin test testacc testrace updatedeps
+6 -2
View File
@@ -24,7 +24,7 @@ from a single source configuration.
Packer is lightweight, runs on every major operating system, and is highly
performant, creating machine images for multiple platforms in parallel. Packer
comes out of the box with support for many platforms, the full list of which can
be found at https://www.packer.io/docs/builders/index.html.
be found at https://www.packer.io/docs/builders/index.html.
Support for other platforms can be added via plugins.
@@ -32,6 +32,10 @@ The images that Packer creates can easily be turned into
[Vagrant](http://www.vagrantup.com) boxes.
## Quick Start
Download and install packages and dependencies
```
go get github.com/hashicorp/packer
```
**Note:** There is a great
[introduction and getting started guide](https://www.packer.io/intro)
@@ -80,7 +84,7 @@ Packer will build an AMI according to the "quick-start" template. The AMI
will be available in your AWS account. To delete the AMI, you must manually
delete it using the [AWS console](https://console.aws.amazon.com/). Packer
builds your images, it does not manage their lifecycle. Where they go, how
they're run, etc., is up to you.
they're run, etc. is up to you.
## Documentation
Vendored
-10
View File
@@ -5,10 +5,6 @@ LINUX_BASE_BOX = "bento/ubuntu-16.04"
FREEBSD_BASE_BOX = "jen20/FreeBSD-12.0-CURRENT"
Vagrant.configure(2) do |config|
if Vagrant.has_plugin?("vagrant-cachier")
config.cache.scope = :box
end
# Compilation and development boxes
config.vm.define "linux", autostart: true, primary: true do |vmCfg|
vmCfg.vm.box = LINUX_BASE_BOX
@@ -73,12 +69,6 @@ def configureProviders(vmCfg, cpus: "2", memory: "2048")
end
end
vmCfg.vm.provider "docker" do |d, override|
d.build_dir = "."
d.has_ssh = true
override.vm.box = nil
end
return vmCfg
end
+1 -1
View File
@@ -27,7 +27,7 @@ build_script:
- git rev-parse HEAD
# go test $(go list ./... | grep -v vendor)
- ps: |
go.exe test -timeout=2m (go.exe list ./... `
go.exe test (go.exe list ./... `
|? { -not $_.Contains('/vendor/') } `
|? { $_ -ne 'github.com/hashicorp/packer/builder/parallels/common' } `
|? { $_ -ne 'github.com/hashicorp/packer/common' }`
+23 -53
View File
@@ -57,11 +57,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return nil, err
}
if b.config.PackerConfig.PackerForce {
b.config.AlicloudImageForceDelete = true
b.config.AlicloudImageForceDeleteSnapshots = true
}
// Accumulate any errors
var errs *packer.MultiError
errs = packer.MultiErrorAppend(errs, b.config.AlicloudAccessConfig.Prepare(&b.config.ctx)...)
@@ -72,7 +67,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AlicloudAccessKey, b.config.AlicloudSecretKey)
log.Println(common.ScrubConfig(b.config, b.config.AlicloudAccessKey, b.config.AlicloudSecretKey))
return nil, nil
}
@@ -83,7 +78,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
return nil, err
}
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("config", b.config)
state.Put("client", client)
state.Put("hook", hook)
state.Put("ui", ui)
@@ -99,11 +94,14 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
&stepCheckAlicloudSourceImage{
SourceECSImageId: b.config.AlicloudSourceImage,
},
&stepConfigAlicloudKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.Comm,
DebugKeyPath: fmt.Sprintf("ecs_%s.pem", b.config.PackerBuildName),
RegionId: b.config.AlicloudRegion,
&StepConfigAlicloudKeyPair{
Debug: b.config.PackerDebug,
KeyPairName: b.config.SSHKeyPairName,
PrivateKeyFile: b.config.Comm.SSHPrivateKey,
TemporaryKeyPairName: b.config.TemporaryKeyPairName,
SSHAgentAuth: b.config.Comm.SSHAgentAuth,
DebugKeyPath: fmt.Sprintf("ecs_%s.pem", b.config.PackerBuildName),
RegionId: b.config.AlicloudRegion,
},
}
if b.chooseNetworkType() == VpcNet {
@@ -138,21 +136,18 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
ZoneId: b.config.ZoneId,
})
if b.chooseNetworkType() == VpcNet {
steps = append(steps, &stepConfigAlicloudEIP{
steps = append(steps, &setpConfigAlicloudEIP{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
RegionId: b.config.AlicloudRegion,
InternetChargeType: b.config.InternetChargeType,
InternetMaxBandwidthOut: b.config.InternetMaxBandwidthOut,
SSHPrivateIp: b.config.SSHPrivateIp,
})
} else {
steps = append(steps, &stepConfigAlicloudPublicIP{
RegionId: b.config.AlicloudRegion,
SSHPrivateIp: b.config.SSHPrivateIp,
RegionId: b.config.AlicloudRegion,
})
}
steps = append(steps,
&stepAttachKeyPair{},
&stepAttachKeyPar{},
&stepRunAlicloudInstance{},
&stepMountAlicloudDisk{},
&communicator.StepConnect{
@@ -160,44 +155,27 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
Host: SSHHost(
client,
b.config.SSHPrivateIp),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
SSHConfig: SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
b.config.RunConfig.Comm.SSHPassword),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&stepStopAlicloudInstance{
ForceStop: b.config.ForceStopInstance,
DisableStop: b.config.DisableStopInstance,
ForceStop: b.config.ForceStopInstance,
},
&stepDeleteAlicloudImageSnapshots{
AlicloudImageForceDeleteSnapshots: b.config.AlicloudImageForceDeleteSnapshots,
AlicloudImageForceDelete: b.config.AlicloudImageForceDelete,
AlicloudImageName: b.config.AlicloudImageName,
AlicloudImageDestinationRegions: b.config.AlicloudImageConfig.AlicloudImageDestinationRegions,
AlicloudImageDestinationNames: b.config.AlicloudImageConfig.AlicloudImageDestinationNames,
})
if b.config.AlicloudImageIgnoreDataDisks {
steps = append(steps, &stepCreateAlicloudSnapshot{
WaitSnapshotReadyTimeout: b.getSnapshotReadyTimeout(),
})
}
steps = append(steps,
&stepCreateAlicloudImage{
AlicloudImageIgnoreDataDisks: b.config.AlicloudImageIgnoreDataDisks,
WaitSnapshotReadyTimeout: b.getSnapshotReadyTimeout(),
},
&stepCreateTags{
Tags: b.config.AlicloudImageTags,
},
&stepRegionCopyAlicloudImage{
&stepCreateAlicloudImage{},
&setpRegionCopyAlicloudImage{
AlicloudImageDestinationRegions: b.config.AlicloudImageDestinationRegions,
AlicloudImageDestinationNames: b.config.AlicloudImageDestinationNames,
RegionId: b.config.AlicloudRegion,
},
&stepShareAlicloudImage{
&setpShareAlicloudImage{
AlicloudImageShareAccounts: b.config.AlicloudImageShareAccounts,
AlicloudImageUNShareAccounts: b.config.AlicloudImageUNShareAccounts,
RegionId: b.config.AlicloudRegion,
@@ -253,7 +231,7 @@ func (b *Builder) isVpcSpecified() bool {
func (b *Builder) isUserDataNeeded() bool {
// Public key setup requires userdata
if b.config.RunConfig.Comm.SSHPrivateKeyFile != "" {
if b.config.RunConfig.Comm.SSHPrivateKey != "" {
return true
}
@@ -261,13 +239,5 @@ func (b *Builder) isUserDataNeeded() bool {
}
func (b *Builder) isKeyPairNeeded() bool {
return b.config.Comm.SSHKeyPairName != "" || b.config.Comm.SSHTemporaryKeyPairName != ""
}
func (b *Builder) getSnapshotReadyTimeout() int {
if b.config.WaitSnapshotReadyTimeout > 0 {
return b.config.WaitSnapshotReadyTimeout
}
return ALICLOUD_DEFAULT_LONG_TIMEOUT
return b.config.SSHKeyPairName != "" || b.config.TemporaryKeyPairName != ""
}
+2 -73
View File
@@ -113,17 +113,6 @@ func TestBuilderAcc_forceDeleteSnapshot(t *testing.T) {
})
}
func TestBuilderAcc_imageTags(t *testing.T) {
builderT.Test(t, builderT.TestCase{
PreCheck: func() {
testAccPreCheck(t)
},
Builder: &Builder{},
Template: testBuilderAccImageTags,
Check: checkImageTags(),
})
}
func checkSnapshotsDeleted(snapshotIds []string) builderT.TestCheckFunc {
return func(artifacts []packer.Artifact) error {
// Verify the snapshots are gone
@@ -218,51 +207,6 @@ func checkRegionCopy(regions []string) builderT.TestCheckFunc {
}
}
func checkImageTags() builderT.TestCheckFunc {
return func(artifacts []packer.Artifact) error {
if len(artifacts) > 1 {
return fmt.Errorf("more than 1 artifact")
}
// Get the actual *Artifact pointer so we can access the AMIs directly
artifactRaw := artifacts[0]
artifact, ok := artifactRaw.(*Artifact)
if !ok {
return fmt.Errorf("unknown artifact: %#v", artifactRaw)
}
// describe the image, get block devices with a snapshot
client, _ := testAliyunClient()
tags, _, err := client.DescribeTags(
&ecs.DescribeTagsArgs{
RegionId: "cn-beijing",
ResourceType: ecs.TagResourceImage,
ResourceId: artifact.AlicloudImages["cn-beijing"],
})
if err != nil {
return fmt.Errorf("Error retrieving Image Attributes for ECS Image Artifact (%#v) "+
"in ECS Image Tags Test: %s", artifact, err)
}
failed := false
if len(tags) != 2 {
failed = true
}
if !failed {
for i := 0; i < len(tags); i++ {
if tags[i].TagKey == "TagKey1" && tags[i].TagValue != "TagValue1" {
failed = true
} else if tags[i].TagKey == "TagKey2" && tags[i].TagValue != "TagValue2" {
failed = true
} else if tags[i].TagKey != "TagKey1" && tags[i].TagKey != "TagKey2" {
failed = true
}
}
}
if failed {
return fmt.Errorf("tags is not correctly set %#v", tags)
}
return nil
}
}
func testAccPreCheck(t *testing.T) {
if v := os.Getenv("ALICLOUD_ACCESS_KEY"); v == "" {
t.Fatal("ALICLOUD_ACCESS_KEY must be set for acceptance tests")
@@ -292,7 +236,8 @@ const testBuilderAccBasic = `
"type": "test",
"region": "cn-beijing",
"instance_type": "ecs.n1.tiny",
"source_image":"ubuntu_16_0402_64_20G_alibase_20180409.vhd",
"source_image":"ubuntu_16_0402_64_40G_base_20170222.vhd",
"ssh_username": "ubuntu",
"io_optimized":"true",
"ssh_username":"root",
"image_name": "packer-test_{{timestamp}}"
@@ -361,22 +306,6 @@ const testBuilderAccSharing = `
}
`
const testBuilderAccImageTags = `
{ "builders": [{
"type": "test",
"region": "cn-beijing",
"instance_type": "ecs.n1.tiny",
"source_image":"ubuntu_16_0402_64_20G_alibase_20180409.vhd",
"ssh_username": "root",
"io_optimized":"true",
"image_name": "packer-test_{{timestamp}}",
"tags": {
"TagKey1": "TagValue1",
"TagKey2": "TagValue2"
}
}]
}`
func buildForceDeregisterConfig(val, name string) string {
return fmt.Sprintf(testBuilderAccForceDelete, val, name)
}
-139
View File
@@ -1,7 +1,6 @@
package ecs
import (
"reflect"
"testing"
"github.com/hashicorp/packer/packer"
@@ -94,141 +93,3 @@ func TestBuilderPrepare_InvalidKey(t *testing.T) {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_Devices(t *testing.T) {
var b Builder
config := testBuilderConfig()
config["system_disk_mapping"] = map[string]interface{}{
"disk_category": "cloud",
"disk_description": "system disk",
"disk_name": "system_disk",
"disk_size": 60,
}
config["image_disk_mappings"] = []map[string]interface{}{
{
"disk_category": "cloud_efficiency",
"disk_name": "data_disk1",
"disk_size": 100,
"disk_snapshot_id": "s-1",
"disk_description": "data disk1",
"disk_device": "/dev/xvdb",
"disk_delete_with_instance": false,
},
{
"disk_name": "data_disk2",
"disk_device": "/dev/xvdc",
},
}
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if !reflect.DeepEqual(b.config.ECSSystemDiskMapping, AlicloudDiskDevice{
DiskCategory: "cloud",
Description: "system disk",
DiskName: "system_disk",
DiskSize: 60,
}) {
t.Fatalf("system disk is not set properly, actual: %#v", b.config.ECSSystemDiskMapping)
}
if !reflect.DeepEqual(b.config.ECSImagesDiskMappings, []AlicloudDiskDevice{
{
DiskCategory: "cloud_efficiency",
DiskName: "data_disk1",
DiskSize: 100,
SnapshotId: "s-1",
Description: "data disk1",
Device: "/dev/xvdb",
DeleteWithInstance: false,
},
{
DiskName: "data_disk2",
Device: "/dev/xvdc",
},
}) {
t.Fatalf("data disks are not set properly, actual: %#v", b.config.ECSImagesDiskMappings)
}
}
func TestBuilderPrepare_IgnoreDataDisks(t *testing.T) {
var b Builder
config := testBuilderConfig()
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.AlicloudImageIgnoreDataDisks != false {
t.Fatalf("image_ignore_data_disks is not set properly, expect: %t, actual: %t", false, b.config.AlicloudImageIgnoreDataDisks)
}
config["image_ignore_data_disks"] = "false"
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.AlicloudImageIgnoreDataDisks != false {
t.Fatalf("image_ignore_data_disks is not set properly, expect: %t, actual: %t", false, b.config.AlicloudImageIgnoreDataDisks)
}
config["image_ignore_data_disks"] = "true"
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.AlicloudImageIgnoreDataDisks != true {
t.Fatalf("image_ignore_data_disks is not set properly, expect: %t, actual: %t", true, b.config.AlicloudImageIgnoreDataDisks)
}
}
func TestBuilderPrepare_WaitSnapshotReadyTimeout(t *testing.T) {
var b Builder
config := testBuilderConfig()
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.WaitSnapshotReadyTimeout != 0 {
t.Fatalf("wait_snapshot_ready_timeout is not set properly, expect: %d, actual: %d", 0, b.config.WaitSnapshotReadyTimeout)
}
if b.getSnapshotReadyTimeout() != ALICLOUD_DEFAULT_LONG_TIMEOUT {
t.Fatalf("default timeout is not set properly, expect: %d, actual: %d", ALICLOUD_DEFAULT_LONG_TIMEOUT, b.getSnapshotReadyTimeout())
}
config["wait_snapshot_ready_timeout"] = ALICLOUD_DEFAULT_TIMEOUT
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.WaitSnapshotReadyTimeout != ALICLOUD_DEFAULT_TIMEOUT {
t.Fatalf("wait_snapshot_ready_timeout is not set properly, expect: %d, actual: %d", ALICLOUD_DEFAULT_TIMEOUT, b.config.WaitSnapshotReadyTimeout)
}
if b.getSnapshotReadyTimeout() != ALICLOUD_DEFAULT_TIMEOUT {
t.Fatalf("default timeout is not set properly, expect: %d, actual: %d", ALICLOUD_DEFAULT_TIMEOUT, b.getSnapshotReadyTimeout())
}
}
+11 -14
View File
@@ -21,24 +21,21 @@ type AlicloudDiskDevice struct {
}
type AlicloudDiskDevices struct {
ECSSystemDiskMapping AlicloudDiskDevice `mapstructure:"system_disk_mapping"`
ECSImagesDiskMappings []AlicloudDiskDevice `mapstructure:"image_disk_mappings"`
}
type AlicloudImageConfig struct {
AlicloudImageName string `mapstructure:"image_name"`
AlicloudImageVersion string `mapstructure:"image_version"`
AlicloudImageDescription string `mapstructure:"image_description"`
AlicloudImageShareAccounts []string `mapstructure:"image_share_account"`
AlicloudImageUNShareAccounts []string `mapstructure:"image_unshare_account"`
AlicloudImageDestinationRegions []string `mapstructure:"image_copy_regions"`
AlicloudImageDestinationNames []string `mapstructure:"image_copy_names"`
AlicloudImageForceDelete bool `mapstructure:"image_force_delete"`
AlicloudImageForceDeleteSnapshots bool `mapstructure:"image_force_delete_snapshots"`
AlicloudImageForceDeleteInstances bool `mapstructure:"image_force_delete_instances"`
AlicloudImageIgnoreDataDisks bool `mapstructure:"image_ignore_data_disks"`
AlicloudImageSkipRegionValidation bool `mapstructure:"skip_region_validation"`
AlicloudImageTags map[string]string `mapstructure:"tags"`
AlicloudImageName string `mapstructure:"image_name"`
AlicloudImageVersion string `mapstructure:"image_version"`
AlicloudImageDescription string `mapstructure:"image_description"`
AlicloudImageShareAccounts []string `mapstructure:"image_share_account"`
AlicloudImageUNShareAccounts []string `mapstructure:"image_unshare_account"`
AlicloudImageDestinationRegions []string `mapstructure:"image_copy_regions"`
AlicloudImageDestinationNames []string `mapstructure:"image_copy_names"`
AlicloudImageForceDelete bool `mapstructure:"image_force_delete"`
AlicloudImageForceDeleteSnapshots bool `mapstructure:"image_force_delete_snapshots"`
AlicloudImageForceDeleteInstances bool `mapstructure:"image_force_delete_instances"`
AlicloudImageSkipRegionValidation bool `mapstructure:"skip_region_validation"`
AlicloudDiskDevices `mapstructure:",squash"`
}
-18
View File
@@ -55,24 +55,6 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
}
func TestECSImageConfigPrepare_imageTags(t *testing.T) {
c := testAlicloudImageConfig()
c.AlicloudImageTags = map[string]string{
"TagKey1": "TagValue1",
"TagKey2": "TagValue2",
}
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if len(c.AlicloudImageTags) != 2 || c.AlicloudImageTags["TagKey1"] != "TagValue1" ||
c.AlicloudImageTags["TagKey2"] != "TagValue2" {
t.Fatalf("invalid value, expected: %s, actual: %s", map[string]string{
"TagKey1": "TagValue1",
"TagKey2": "TagValue2",
}, c.AlicloudImageTags)
}
}
func regionsToString() []string {
var regions []string
for _, region := range common.ValidRegions {
-12
View File
@@ -20,15 +20,3 @@ func message(state multistep.StateBag, module string) {
}
}
func halt(state multistep.StateBag, err error, prefix string) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if prefix != "" {
err = fmt.Errorf("%s: %s", prefix, err)
}
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
+7 -7
View File
@@ -19,7 +19,6 @@ type RunConfig struct {
Description string `mapstructure:"description"`
AlicloudSourceImage string `mapstructure:"source_image"`
ForceStopInstance bool `mapstructure:"force_stop_instance"`
DisableStopInstance bool `mapstructure:"disable_stop_instance"`
SecurityGroupId string `mapstructure:"security_group_id"`
SecurityGroupName string `mapstructure:"security_group_name"`
UserData string `mapstructure:"user_data"`
@@ -32,18 +31,19 @@ type RunConfig struct {
InstanceName string `mapstructure:"instance_name"`
InternetChargeType string `mapstructure:"internet_charge_type"`
InternetMaxBandwidthOut int `mapstructure:"internet_max_bandwidth_out"`
WaitSnapshotReadyTimeout int `mapstructure:"wait_snapshot_ready_timeout"`
TemporaryKeyPairName string `mapstructure:"temporary_key_pair_name"`
// Communicator settings
Comm communicator.Config `mapstructure:",squash"`
SSHPrivateIp bool `mapstructure:"ssh_private_ip"`
Comm communicator.Config `mapstructure:",squash"`
SSHKeyPairName string `mapstructure:"ssh_keypair_name"`
SSHPrivateIp bool `mapstructure:"ssh_private_ip"`
}
func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
if c.Comm.SSHKeyPairName == "" && c.Comm.SSHTemporaryKeyPairName == "" &&
c.Comm.SSHPrivateKeyFile == "" && c.Comm.SSHPassword == "" && c.Comm.WinRMPassword == "" {
if c.SSHKeyPairName == "" && c.TemporaryKeyPairName == "" &&
c.Comm.SSHPrivateKey == "" && c.Comm.SSHPassword == "" && c.Comm.WinRMPassword == "" {
c.Comm.SSHTemporaryKeyPairName = fmt.Sprintf("packer_%s", uuid.TimeOrderedUUID())
c.TemporaryKeyPairName = fmt.Sprintf("packer_%s", uuid.TimeOrderedUUID())
}
// Validation
+4 -58
View File
@@ -2,7 +2,6 @@ package ecs
import (
"io/ioutil"
"os"
"testing"
"github.com/hashicorp/packer/helper/communicator"
@@ -69,7 +68,6 @@ func TestRunConfigPrepare_UserData(t *testing.T) {
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserData = "foo"
@@ -94,7 +92,6 @@ func TestRunConfigPrepare_UserDataFile(t *testing.T) {
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserDataFile = tf.Name()
@@ -105,72 +102,21 @@ func TestRunConfigPrepare_UserDataFile(t *testing.T) {
func TestRunConfigPrepare_TemporaryKeyPairName(t *testing.T) {
c := testConfig()
c.Comm.SSHTemporaryKeyPairName = ""
c.TemporaryKeyPairName = ""
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.Comm.SSHTemporaryKeyPairName == "" {
if c.TemporaryKeyPairName == "" {
t.Fatal("keypair name is empty")
}
c.Comm.SSHTemporaryKeyPairName = "ssh-key-123"
c.TemporaryKeyPairName = "ssh-key-123"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.Comm.SSHTemporaryKeyPairName != "ssh-key-123" {
if c.TemporaryKeyPairName != "ssh-key-123" {
t.Fatal("keypair name does not match")
}
}
func TestRunConfigPrepare_SSHPrivateIp(t *testing.T) {
c := testConfig()
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.SSHPrivateIp != false {
t.Fatalf("invalid value, expected: %t, actul: %t", false, c.SSHPrivateIp)
}
c.SSHPrivateIp = true
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.SSHPrivateIp != true {
t.Fatalf("invalid value, expected: %t, actul: %t", true, c.SSHPrivateIp)
}
c.SSHPrivateIp = false
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.SSHPrivateIp != false {
t.Fatalf("invalid value, expected: %t, actul: %t", false, c.SSHPrivateIp)
}
}
func TestRunConfigPrepare_DisableStopInstance(t *testing.T) {
c := testConfig()
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.DisableStopInstance != false {
t.Fatalf("invalid value, expected: %t, actul: %t", false, c.DisableStopInstance)
}
c.DisableStopInstance = true
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.DisableStopInstance != true {
t.Fatalf("invalid value, expected: %t, actul: %t", true, c.DisableStopInstance)
}
c.DisableStopInstance = false
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.DisableStopInstance != false {
t.Fatalf("invalid value, expected: %t, actul: %t", false, c.DisableStopInstance)
}
}
+60
View File
@@ -1,9 +1,15 @@
package ecs
import (
"fmt"
"net"
"os"
"time"
packerssh "github.com/hashicorp/packer/communicator/ssh"
"github.com/hashicorp/packer/helper/multistep"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/agent"
)
var (
@@ -21,3 +27,57 @@ func SSHHost(e alicloudSSHHelper, private bool) func(multistep.StateBag) (string
return ipAddress, nil
}
}
// SSHConfig returns a function that can be used for the SSH communicator
// config for connecting to the instance created over SSH using the private key
// or password.
func SSHConfig(useAgent bool, username, password string) func(multistep.StateBag) (*ssh.ClientConfig, error) {
return func(state multistep.StateBag) (*ssh.ClientConfig, error) {
if useAgent {
authSock := os.Getenv("SSH_AUTH_SOCK")
if authSock == "" {
return nil, fmt.Errorf("SSH_AUTH_SOCK is not set")
}
sshAgent, err := net.Dial("unix", authSock)
if err != nil {
return nil, fmt.Errorf("Cannot connect to SSH Agent socket %q: %s", authSock, err)
}
return &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{
ssh.PublicKeysCallback(agent.NewClient(sshAgent).Signers),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}, nil
}
privateKey, hasKey := state.GetOk("privateKey")
if hasKey {
signer, err := ssh.ParsePrivateKey([]byte(privateKey.(string)))
if err != nil {
return nil, fmt.Errorf("Error setting up SSH config: %s", err)
}
return &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{
ssh.PublicKeys(signer),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}, nil
} else {
return &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{
ssh.Password(password),
ssh.KeyboardInteractive(
packerssh.PasswordKeyboardInteractive(password)),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}, nil
}
}
}
+14 -14
View File
@@ -12,19 +12,19 @@ import (
"github.com/hashicorp/packer/packer"
)
type stepAttachKeyPair struct {
type stepAttachKeyPar struct {
}
func (s *stepAttachKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(*Config)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
timeoutPoint := time.Now().Add(120 * time.Second)
keyPairName := config.Comm.SSHKeyPairName
func (s *stepAttachKeyPar) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
keyPairName := state.Get("keyPair").(string)
if keyPairName == "" {
return multistep.ActionContinue
}
ui := state.Get("ui").(packer.Ui)
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(Config)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
timeoutPoint := time.Now().Add(120 * time.Second)
for {
err := client.AttachKeyPair(&ecs.AttachKeyPairArgs{RegionId: common.Region(config.AlicloudRegion),
KeyPairName: keyPairName, InstanceIds: "[\"" + instance.InstanceId + "\"]"})
@@ -50,15 +50,15 @@ func (s *stepAttachKeyPair) Run(_ context.Context, state multistep.StateBag) mul
return multistep.ActionContinue
}
func (s *stepAttachKeyPair) Cleanup(state multistep.StateBag) {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(*Config)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
keyPairName := config.Comm.SSHKeyPairName
func (s *stepAttachKeyPar) Cleanup(state multistep.StateBag) {
keyPairName := state.Get("keyPair").(string)
if keyPairName == "" {
return
}
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
err := client.DetachKeyPair(&ecs.DetachKeyPairArgs{RegionId: common.Region(config.AlicloudRegion),
KeyPairName: keyPairName, InstanceIds: "[\"" + instance.InstanceId + "\"]"})
@@ -16,14 +16,10 @@ type stepCheckAlicloudSourceImage struct {
func (s *stepCheckAlicloudSourceImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)
args := &ecs.DescribeImagesArgs{
RegionId: common.Region(config.AlicloudRegion),
ImageId: config.AlicloudSourceImage,
}
args.PageSize = 50
images, _, err := client.DescribeImages(args)
images, _, err := client.DescribeImages(&ecs.DescribeImagesArgs{RegionId: common.Region(config.AlicloudRegion),
ImageId: config.AlicloudSourceImage})
if err != nil {
err := fmt.Errorf("Error querying alicloud image: %s", err)
state.Put("error", err)
@@ -31,19 +27,6 @@ func (s *stepCheckAlicloudSourceImage) Run(_ context.Context, state multistep.St
return multistep.ActionHalt
}
// Describe markerplace image
args.ImageOwnerAlias = ecs.ImageOwnerMarketplace
imageMarkets, _, err := client.DescribeImages(args)
if err != nil {
err := fmt.Errorf("Error querying alicloud marketplace image: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(imageMarkets) > 0 {
images = append(images, imageMarkets...)
}
if len(images) == 0 {
err := fmt.Errorf("No alicloud image was found matching filters: %v", config.AlicloudSourceImage)
state.Put("error", err)
+3 -17
View File
@@ -10,34 +10,20 @@ import (
"github.com/hashicorp/packer/packer"
)
type stepConfigAlicloudEIP struct {
type setpConfigAlicloudEIP struct {
AssociatePublicIpAddress bool
RegionId string
InternetChargeType string
InternetMaxBandwidthOut int
allocatedId string
SSHPrivateIp bool
}
func (s *stepConfigAlicloudEIP) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
func (s *setpConfigAlicloudEIP) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
if s.SSHPrivateIp {
ipaddress := instance.VpcAttributes.PrivateIpAddress.IpAddress
if len(ipaddress) == 0 {
ui.Say("Failed to get private ip of instance")
return multistep.ActionHalt
}
state.Put("ipaddress", ipaddress[0])
return multistep.ActionContinue
}
ui.Say("Allocating eip")
ipaddress, allocateId, err := client.AllocateEipAddress(&ecs.AllocateEipAddressArgs{
RegionId: common.Region(s.RegionId), InternetChargeType: common.InternetChargeType(s.InternetChargeType),
Bandwidth: s.InternetMaxBandwidthOut,
})
if err != nil {
state.Put("error", err)
@@ -69,7 +55,7 @@ func (s *stepConfigAlicloudEIP) Run(_ context.Context, state multistep.StateBag)
return multistep.ActionContinue
}
func (s *stepConfigAlicloudEIP) Cleanup(state multistep.StateBag) {
func (s *setpConfigAlicloudEIP) Cleanup(state multistep.StateBag) {
if len(s.allocatedId) == 0 {
return
}
+29 -23
View File
@@ -3,62 +3,68 @@ package ecs
import (
"context"
"fmt"
"io/ioutil"
"os"
"runtime"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
type stepConfigAlicloudKeyPair struct {
Debug bool
Comm *communicator.Config
DebugKeyPath string
RegionId string
type StepConfigAlicloudKeyPair struct {
Debug bool
SSHAgentAuth bool
DebugKeyPath string
TemporaryKeyPairName string
KeyPairName string
PrivateKeyFile string
RegionId string
keyName string
}
func (s *stepConfigAlicloudKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepConfigAlicloudKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if s.Comm.SSHPrivateKeyFile != "" {
if s.PrivateKeyFile != "" {
ui.Say("Using existing SSH private key")
privateKeyBytes, err := s.Comm.ReadSSHPrivateKeyFile()
privateKeyBytes, err := ioutil.ReadFile(s.PrivateKeyFile)
if err != nil {
state.Put("error", err)
state.Put("error", fmt.Errorf(
"Error loading configured private key file: %s", err))
return multistep.ActionHalt
}
s.Comm.SSHPrivateKey = privateKeyBytes
state.Put("keyPair", s.KeyPairName)
state.Put("privateKey", string(privateKeyBytes))
return multistep.ActionContinue
}
if s.Comm.SSHAgentAuth && s.Comm.SSHKeyPairName == "" {
if s.SSHAgentAuth && s.KeyPairName == "" {
ui.Say("Using SSH Agent with key pair in source image")
return multistep.ActionContinue
}
if s.Comm.SSHAgentAuth && s.Comm.SSHKeyPairName != "" {
ui.Say(fmt.Sprintf("Using SSH Agent for existing key pair %s", s.Comm.SSHKeyPairName))
if s.SSHAgentAuth && s.KeyPairName != "" {
ui.Say(fmt.Sprintf("Using SSH Agent for existing key pair %s", s.KeyPairName))
state.Put("keyPair", s.KeyPairName)
return multistep.ActionContinue
}
if s.Comm.SSHTemporaryKeyPairName == "" {
if s.TemporaryKeyPairName == "" {
ui.Say("Not using temporary keypair")
s.Comm.SSHKeyPairName = ""
state.Put("keyPair", "")
return multistep.ActionContinue
}
client := state.Get("client").(*ecs.Client)
ui.Say(fmt.Sprintf("Creating temporary keypair: %s", s.Comm.SSHTemporaryKeyPairName))
ui.Say(fmt.Sprintf("Creating temporary keypair: %s", s.TemporaryKeyPairName))
keyResp, err := client.CreateKeyPair(&ecs.CreateKeyPairArgs{
KeyPairName: s.Comm.SSHTemporaryKeyPairName,
KeyPairName: s.TemporaryKeyPairName,
RegionId: common.Region(s.RegionId),
})
if err != nil {
@@ -67,11 +73,11 @@ func (s *stepConfigAlicloudKeyPair) Run(_ context.Context, state multistep.State
}
// Set the keyname so we know to delete it later
s.keyName = s.Comm.SSHTemporaryKeyPairName
s.keyName = s.TemporaryKeyPairName
// Set some state data for use in future steps
s.Comm.SSHKeyPairName = s.keyName
s.Comm.SSHPrivateKey = []byte(keyResp.PrivateKeyBody)
state.Put("keyPair", s.keyName)
state.Put("privateKey", keyResp.PrivateKeyBody)
// If we're in debug mode, output the private key to the working
// directory.
@@ -102,11 +108,11 @@ func (s *stepConfigAlicloudKeyPair) Run(_ context.Context, state multistep.State
return multistep.ActionContinue
}
func (s *stepConfigAlicloudKeyPair) Cleanup(state multistep.StateBag) {
func (s *StepConfigAlicloudKeyPair) Cleanup(state multistep.StateBag) {
// If no key name is set, then we never created it, so just return
// If we used an SSH private key file, do not go about deleting
// keypairs
if s.Comm.SSHPrivateKeyFile != "" || (s.Comm.SSHKeyPairName == "" && s.keyName == "") {
if s.PrivateKeyFile != "" || (s.KeyPairName == "" && s.keyName == "") {
return
}
@@ -12,7 +12,6 @@ import (
type stepConfigAlicloudPublicIP struct {
publicIPAddress string
RegionId string
SSHPrivateIp bool
}
func (s *stepConfigAlicloudPublicIP) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
@@ -20,16 +19,6 @@ func (s *stepConfigAlicloudPublicIP) Run(_ context.Context, state multistep.Stat
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
if s.SSHPrivateIp {
ipaddress := instance.InnerIpAddress.IpAddress
if len(ipaddress) == 0 {
ui.Say("Failed to get private ip of instance")
return multistep.ActionHalt
}
state.Put("ipaddress", ipaddress[0])
return multistep.ActionContinue
}
ipaddress, err := client.AllocatePublicIpAddress(instance.InstanceId)
if err != nil {
state.Put("error", err)
+2 -3
View File
@@ -20,7 +20,7 @@ type stepConfigAlicloudVPC struct {
}
func (s *stepConfigAlicloudVPC) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
@@ -85,8 +85,7 @@ func (s *stepConfigAlicloudVPC) Cleanup(state multistep.StateBag) {
e, _ := err.(*common.Error)
if (e.Code == "DependencyViolation.Instance" || e.Code == "DependencyViolation.RouteEntry" ||
e.Code == "DependencyViolation.VSwitch" ||
e.Code == "DependencyViolation.SecurityGroup" ||
e.Code == "Forbbiden") && time.Now().Before(timeoutPoint) {
e.Code == "DependencyViolation.SecurityGroup") && time.Now().Before(timeoutPoint) {
time.Sleep(1 * time.Second)
continue
}
+1 -1
View File
@@ -24,7 +24,7 @@ func (s *stepConfigAlicloudVSwitch) Run(_ context.Context, state multistep.State
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
vpcId := state.Get("vpcid").(string)
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
if len(s.VSwitchId) != 0 {
vswitchs, _, err := client.DescribeVSwitches(&ecs.DescribeVSwitchesArgs{
+28 -35
View File
@@ -11,13 +11,11 @@ import (
)
type stepCreateAlicloudImage struct {
AlicloudImageIgnoreDataDisks bool
WaitSnapshotReadyTimeout int
image *ecs.ImageType
image *ecs.ImageType
}
func (s *stepCreateAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
@@ -26,53 +24,48 @@ func (s *stepCreateAlicloudImage) Run(_ context.Context, state multistep.StateBa
var imageId string
var err error
if s.AlicloudImageIgnoreDataDisks {
snapshotId := state.Get("alicloudsnapshot").(string)
imageId, err = client.CreateImage(&ecs.CreateImageArgs{
RegionId: common.Region(config.AlicloudRegion),
SnapshotId: snapshotId,
ImageName: config.AlicloudImageName,
ImageVersion: config.AlicloudImageVersion,
Description: config.AlicloudImageDescription})
} else {
instance := state.Get("instance").(*ecs.InstanceAttributesType)
imageId, err = client.CreateImage(&ecs.CreateImageArgs{
RegionId: common.Region(config.AlicloudRegion),
InstanceId: instance.InstanceId,
ImageName: config.AlicloudImageName,
ImageVersion: config.AlicloudImageVersion,
Description: config.AlicloudImageDescription})
}
instance := state.Get("instance").(*ecs.InstanceAttributesType)
imageId, err = client.CreateImage(&ecs.CreateImageArgs{
RegionId: common.Region(config.AlicloudRegion),
InstanceId: instance.InstanceId,
ImageName: config.AlicloudImageName,
ImageVersion: config.AlicloudImageVersion,
Description: config.AlicloudImageDescription})
if err != nil {
return halt(state, err, "Error creating image")
err := fmt.Errorf("Error creating image: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
err = client.WaitForImageReady(common.Region(config.AlicloudRegion), imageId, s.WaitSnapshotReadyTimeout)
err = client.WaitForImageReady(common.Region(config.AlicloudRegion),
imageId, ALICLOUD_DEFAULT_LONG_TIMEOUT)
if err != nil {
return halt(state, err, "Timeout waiting for image to be created")
err := fmt.Errorf("Timeout waiting for image to be created: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
images, _, err := client.DescribeImages(&ecs.DescribeImagesArgs{
RegionId: common.Region(config.AlicloudRegion),
ImageId: imageId})
if err != nil {
return halt(state, err, "Error querying created imaged")
err := fmt.Errorf("Error querying created image: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(images) == 0 {
return halt(state, err, "Unable to find created image")
err := fmt.Errorf("Unable to find created image: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
s.image = &images[0]
var snapshotIds = []string{}
for _, device := range images[0].DiskDeviceMappings.DiskDeviceMapping {
snapshotIds = append(snapshotIds, device.SnapshotId)
}
state.Put("alicloudimage", imageId)
state.Put("alicloudsnapshots", snapshotIds)
alicloudImages := make(map[string]string)
alicloudImages[config.AlicloudRegion] = images[0].ImageId
state.Put("alicloudimages", alicloudImages)
@@ -92,7 +85,7 @@ func (s *stepCreateAlicloudImage) Cleanup(state multistep.StateBag) {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
ui.Say("Deleting the image because of cancellation or error...")
if err := client.DeleteImage(common.Region(config.AlicloudRegion), s.image.ImageId); err != nil {
+1 -11
View File
@@ -28,7 +28,7 @@ type stepCreateAlicloudInstance struct {
func (s *stepCreateAlicloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)
source_image := state.Get("source_image").(*ecs.ImageType)
network_type := state.Get("networktype").(InstanceNetWork)
@@ -66,7 +66,6 @@ func (s *stepCreateAlicloudInstance) Run(_ context.Context, state multistep.Stat
InstanceName: s.InstanceName,
Password: password,
ZoneId: s.ZoneId,
SystemDisk: systemDeviceToDiskType(config.AlicloudImageConfig.ECSSystemDiskMapping),
DataDisk: diskDeviceToDiskType(config.AlicloudImageConfig.ECSImagesDiskMappings),
})
if err != nil {
@@ -148,15 +147,6 @@ func (s *stepCreateAlicloudInstance) getUserData(state multistep.StateBag) (stri
}
func systemDeviceToDiskType(systemDisk AlicloudDiskDevice) ecs.SystemDiskType {
return ecs.SystemDiskType{
DiskName: systemDisk.DiskName,
Category: ecs.DiskCategory(systemDisk.DiskCategory),
Size: systemDisk.DiskSize,
Description: systemDisk.Description,
}
}
func diskDeviceToDiskType(diskDevices []AlicloudDiskDevice) []ecs.DataDiskType {
result := make([]ecs.DataDiskType, len(diskDevices))
for _, diskDevice := range diskDevices {
@@ -1,88 +0,0 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
type stepCreateAlicloudSnapshot struct {
snapshot *ecs.SnapshotType
WaitSnapshotReadyTimeout int
}
func (s *stepCreateAlicloudSnapshot) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
disks, _, err := client.DescribeDisks(&ecs.DescribeDisksArgs{
RegionId: common.Region(config.AlicloudRegion),
InstanceId: instance.InstanceId,
DiskType: ecs.DiskTypeAllSystem,
})
if err != nil {
return halt(state, err, "Error describe disks")
}
if len(disks) == 0 {
return halt(state, err, "Unable to find system disk of instance")
}
// Create the alicloud snapshot
ui.Say(fmt.Sprintf("Creating snapshot from system disk: %s", disks[0].DiskId))
snapshotId, err := client.CreateSnapshot(&ecs.CreateSnapshotArgs{
DiskId: disks[0].DiskId,
})
if err != nil {
return halt(state, err, "Error creating snapshot")
}
err = client.WaitForSnapShotReady(common.Region(config.AlicloudRegion), snapshotId, s.WaitSnapshotReadyTimeout)
if err != nil {
return halt(state, err, "Timeout waiting for snapshot to be created")
}
snapshots, _, err := client.DescribeSnapshots(&ecs.DescribeSnapshotsArgs{
RegionId: common.Region(config.AlicloudRegion),
SnapshotIds: []string{snapshotId},
})
if err != nil {
return halt(state, err, "Error querying created snapshot")
}
if len(snapshots) == 0 {
return halt(state, err, "Unable to find created snapshot")
}
s.snapshot = &snapshots[0]
state.Put("alicloudsnapshot", snapshotId)
return multistep.ActionContinue
}
func (s *stepCreateAlicloudSnapshot) Cleanup(state multistep.StateBag) {
if s.snapshot == nil {
return
}
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if !cancelled && !halted {
return
}
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
ui.Say("Deleting the snapshot because of cancellation or error...")
if err := client.DeleteSnapshot(s.snapshot.SnapshotId); err != nil {
ui.Error(fmt.Sprintf("Error deleting snapshot, it may still be around: %s", err))
return
}
}
-54
View File
@@ -1,54 +0,0 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
type stepCreateTags struct {
Tags map[string]string
}
func (s *stepCreateTags) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
imageId := state.Get("alicloudimage").(string)
snapshotIds := state.Get("alicloudsnapshots").([]string)
if len(s.Tags) == 0 {
return multistep.ActionContinue
}
ui.Say(fmt.Sprintf("Adding tags(%s) to image: %s", s.Tags, imageId))
err := client.AddTags(&ecs.AddTagsArgs{
ResourceId: imageId,
ResourceType: ecs.TagResourceImage,
RegionId: common.Region(config.AlicloudRegion),
Tag: s.Tags,
})
if err != nil {
return halt(state, err, "Error Adding tags to image")
}
for _, snapshotId := range snapshotIds {
ui.Say(fmt.Sprintf("Adding tags(%s) to snapshot: %s", s.Tags, snapshotId))
err = client.AddTags(&ecs.AddTagsArgs{
ResourceId: snapshotId,
ResourceType: ecs.TagResourceSnapshot,
RegionId: common.Region(config.AlicloudRegion),
Tag: s.Tags,
})
if err != nil {
return halt(state, err, "Error Adding tags to snapshot")
}
}
return multistep.ActionContinue
}
func (s *stepCreateTags) Cleanup(state multistep.StateBag) {
// Nothing need to do, tags will be cleaned when the resource is cleaned
}
@@ -15,82 +15,50 @@ type stepDeleteAlicloudImageSnapshots struct {
AlicloudImageForceDelete bool
AlicloudImageForceDeleteSnapshots bool
AlicloudImageName string
AlicloudImageDestinationRegions []string
AlicloudImageDestinationNames []string
}
func (s *stepDeleteAlicloudImageSnapshots) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
config := state.Get("config").(Config)
ui.Say("Deleting image snapshots.")
// Check for force delete
if s.AlicloudImageForceDelete {
err := s.deleteImageAndSnapshots(state, s.AlicloudImageName, config.AlicloudRegion)
if err != nil {
return halt(state, err, "")
}
numberOfName := len(s.AlicloudImageDestinationNames)
if numberOfName == 0 {
images, _, err := client.DescribeImages(&ecs.DescribeImagesArgs{
RegionId: common.Region(config.AlicloudRegion),
ImageName: s.AlicloudImageName,
})
if len(images) < 1 {
return multistep.ActionContinue
}
for index, destinationRegion := range s.AlicloudImageDestinationRegions {
if destinationRegion == config.AlicloudRegion {
for _, image := range images {
if image.ImageOwnerAlias != string(ecs.ImageOwnerSelf) {
log.Printf("You can only delete instances based on customized images %s ", image.ImageId)
continue
}
if index < numberOfName {
err = s.deleteImageAndSnapshots(state, s.AlicloudImageDestinationNames[index], destinationRegion)
if err != nil {
return halt(state, err, "")
err = client.DeleteImage(common.Region(config.AlicloudRegion), image.ImageId)
if err != nil {
err := fmt.Errorf("Failed to delete image: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if s.AlicloudImageForceDeleteSnapshots {
for _, diskDevice := range image.DiskDeviceMappings.DiskDeviceMapping {
if err := client.DeleteSnapshot(diskDevice.SnapshotId); err != nil {
err := fmt.Errorf("Deleting ECS snapshot failed: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
}
} else {
break
}
}
}
return multistep.ActionContinue
}
func (s *stepDeleteAlicloudImageSnapshots) deleteImageAndSnapshots(state multistep.StateBag, imageName string, region string) error {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
images, _, err := client.DescribeImages(&ecs.DescribeImagesArgs{
RegionId: common.Region(region),
ImageName: imageName,
})
if len(images) < 1 {
return nil
}
ui.Say(fmt.Sprintf("Deleting duplicated image and snapshot in %s: %s", region, imageName))
for _, image := range images {
if image.ImageOwnerAlias != string(ecs.ImageOwnerSelf) {
log.Printf("You can not delete non-customized images: %s ", image.ImageId)
continue
}
err = client.DeleteImage(common.Region(region), image.ImageId)
if err != nil {
err := fmt.Errorf("Failed to delete image: %s", err)
return err
}
if s.AlicloudImageForceDeleteSnapshots {
for _, diskDevice := range image.DiskDeviceMappings.DiskDeviceMapping {
if err := client.DeleteSnapshot(diskDevice.SnapshotId); err != nil {
err := fmt.Errorf("Deleting ECS snapshot failed: %s", err)
return err
}
}
}
}
return nil
}
func (s *stepDeleteAlicloudImageSnapshots) Cleanup(state multistep.StateBag) {
}
+1 -1
View File
@@ -14,7 +14,7 @@ type stepMountAlicloudDisk struct {
func (s *stepMountAlicloudDisk) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
alicloudDiskDevices := config.ECSImagesDiskMappings
+1 -1
View File
@@ -23,7 +23,7 @@ func (s *stepPreValidate) Run(_ context.Context, state multistep.StateBag) multi
}
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(*Config)
config := state.Get("config").(Config)
ui.Say("Prevalidating image name...")
images, _, err := client.DescribeImages(&ecs.DescribeImagesArgs{
ImageName: s.AlicloudDestImageName,
@@ -10,13 +10,13 @@ import (
"github.com/hashicorp/packer/packer"
)
type stepRegionCopyAlicloudImage struct {
type setpRegionCopyAlicloudImage struct {
AlicloudImageDestinationRegions []string
AlicloudImageDestinationNames []string
RegionId string
}
func (s *stepRegionCopyAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
func (s *setpRegionCopyAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
if len(s.AlicloudImageDestinationRegions) == 0 {
return multistep.ActionContinue
}
@@ -52,7 +52,7 @@ func (s *stepRegionCopyAlicloudImage) Run(_ context.Context, state multistep.Sta
return multistep.ActionContinue
}
func (s *stepRegionCopyAlicloudImage) Cleanup(state multistep.StateBag) {
func (s *setpRegionCopyAlicloudImage) Cleanup(state multistep.StateBag) {
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {
+1 -3
View File
@@ -24,9 +24,7 @@ func (s *stepRunAlicloudInstance) Run(_ context.Context, state multistep.StateBa
ui.Error(err.Error())
return multistep.ActionHalt
}
ui.Say(fmt.Sprintf("Starting instance: %s", instance.InstanceId))
ui.Say("Starting instance.")
err = client.WaitForInstance(instance.InstanceId, ecs.Running, ALICLOUD_DEFAULT_TIMEOUT)
if err != nil {
err := fmt.Errorf("Timeout waiting for instance to start: %s", err)
+3 -3
View File
@@ -10,13 +10,13 @@ import (
"github.com/hashicorp/packer/packer"
)
type stepShareAlicloudImage struct {
type setpShareAlicloudImage struct {
AlicloudImageShareAccounts []string
AlicloudImageUNShareAccounts []string
RegionId string
}
func (s *stepShareAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
func (s *setpShareAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
alicloudImages := state.Get("alicloudimages").(map[string]string)
@@ -37,7 +37,7 @@ func (s *stepShareAlicloudImage) Run(_ context.Context, state multistep.StateBag
return multistep.ActionContinue
}
func (s *stepShareAlicloudImage) Cleanup(state multistep.StateBag) {
func (s *setpShareAlicloudImage) Cleanup(state multistep.StateBag) {
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {
+4 -8
View File
@@ -10,8 +10,7 @@ import (
)
type stepStopAlicloudInstance struct {
ForceStop bool
DisableStop bool
ForceStop bool
}
func (s *stepStopAlicloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
@@ -19,9 +18,8 @@ func (s *stepStopAlicloudInstance) Run(_ context.Context, state multistep.StateB
instance := state.Get("instance").(*ecs.InstanceAttributesType)
ui := state.Get("ui").(packer.Ui)
if !s.DisableStop {
ui.Say(fmt.Sprintf("Stopping instance: %s", instance.InstanceId))
err := client.StopInstance(instance.InstanceId, s.ForceStop)
err := client.StopInstance(instance.InstanceId, s.ForceStop)
if err != nil {
if err != nil {
err := fmt.Errorf("Error stopping alicloud instance: %s", err)
state.Put("error", err)
@@ -30,9 +28,7 @@ func (s *stepStopAlicloudInstance) Run(_ context.Context, state multistep.StateB
}
}
ui.Say(fmt.Sprintf("Waiting instance stopped: %s", instance.InstanceId))
err := client.WaitForInstance(instance.InstanceId, ecs.Stopped, ALICLOUD_DEFAULT_TIMEOUT)
err = client.WaitForInstance(instance.InstanceId, ecs.Stopped, ALICLOUD_DEFAULT_TIMEOUT)
if err != nil {
err := fmt.Errorf("Error waiting for alicloud instance to stop: %s", err)
state.Put("error", err)
+5 -17
View File
@@ -9,11 +9,9 @@ import (
"log"
"runtime"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
@@ -35,19 +33,16 @@ type Config struct {
CommandWrapper string `mapstructure:"command_wrapper"`
CopyFiles []string `mapstructure:"copy_files"`
DevicePath string `mapstructure:"device_path"`
NVMEDevicePath string `mapstructure:"nvme_device_path"`
FromScratch bool `mapstructure:"from_scratch"`
MountOptions []string `mapstructure:"mount_options"`
MountPartition string `mapstructure:"mount_partition"`
MountPartition int `mapstructure:"mount_partition"`
MountPath string `mapstructure:"mount_path"`
PostMountCommands []string `mapstructure:"post_mount_commands"`
PreMountCommands []string `mapstructure:"pre_mount_commands"`
RootDeviceName string `mapstructure:"root_device_name"`
RootVolumeSize int64 `mapstructure:"root_volume_size"`
RootVolumeType string `mapstructure:"root_volume_type"`
SourceAmi string `mapstructure:"source_ami"`
SourceAmiFilter awscommon.AmiFilterOptions `mapstructure:"source_ami_filter"`
RootVolumeTags awscommon.TagMap `mapstructure:"root_volume_tags"`
ctx interpolate.Context
}
@@ -71,7 +66,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"ami_description",
"snapshot_tags",
"tags",
"root_volume_tags",
"command_wrapper",
"post_mount_commands",
"pre_mount_commands",
@@ -118,8 +112,8 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
b.config.MountPath = "/mnt/packer-amazon-chroot-volumes/{{.Device}}"
}
if b.config.MountPartition == "" {
b.config.MountPartition = "1"
if b.config.MountPartition == 0 {
b.config.MountPartition = 1
}
// Accumulate any errors or warnings
@@ -179,7 +173,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return warns, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return warns, nil
}
@@ -192,9 +186,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if err != nil {
return nil, err
}
ec2conn := ec2.New(session, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
})
ec2conn := ec2.New(session)
wrappedCommand := func(command string) (string, error) {
ctx := b.config.ctx
@@ -227,7 +219,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
EnableAMIENASupport: b.config.AMIENASupport,
AmiFilters: b.config.SourceAmiFilter,
AMIVirtType: b.config.AMIVirtType,
},
&StepCheckRootDevice{},
)
@@ -237,10 +228,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
&StepFlock{},
&StepPrepareDevice{},
&StepCreateVolume{
RootVolumeType: b.config.RootVolumeType,
RootVolumeSize: b.config.RootVolumeSize,
RootVolumeTags: b.config.RootVolumeTags,
Ctx: b.config.ctx,
},
&StepAttachVolume{},
&StepEarlyUnflock{},
-3
View File
@@ -11,8 +11,6 @@ func testConfig() map[string]interface{} {
"ami_name": "foo",
"source_ami": "foo",
"region": "us-east-1",
// region validation logic is checked in ami_config_test
"skip_region_validation": true,
}
}
@@ -30,7 +28,6 @@ func TestBuilderPrepare_AMIName(t *testing.T) {
// Test good
config["ami_name"] = "foo"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
+3 -6
View File
@@ -4,16 +4,15 @@ import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"syscall"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/packer/tmp"
)
// Communicator is a special communicator that works by executing
@@ -24,10 +23,8 @@ type Communicator struct {
}
func (c *Communicator) Start(cmd *packer.RemoteCmd) error {
// need extra escapes for the command since we're wrapping it in quotes
cmd.Command = strconv.Quote(cmd.Command)
command, err := c.CmdWrapper(
fmt.Sprintf("chroot %s /bin/sh -c %s", c.Chroot, cmd.Command))
fmt.Sprintf("chroot %s /bin/sh -c \"%s\"", c.Chroot, cmd.Command))
if err != nil {
return err
}
@@ -67,7 +64,7 @@ func (c *Communicator) Start(cmd *packer.RemoteCmd) error {
func (c *Communicator) Upload(dst string, r io.Reader, fi *os.FileInfo) error {
dst = filepath.Join(c.Chroot, dst)
log.Printf("Uploading to chroot dir: %s", dst)
tf, err := tmp.File("packer-amazon-chroot")
tf, err := ioutil.TempFile("", "packer-amazon-chroot")
if err != nil {
return fmt.Errorf("Error preparing shell script: %s", err)
}
+2 -4
View File
@@ -3,8 +3,8 @@ package chroot
import (
"fmt"
sl "github.com/hashicorp/packer/common/shell-local"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/post-processor/shell-local"
"github.com/hashicorp/packer/template/interpolate"
)
@@ -21,9 +21,7 @@ func RunLocalCommands(commands []string, wrappedCommand CommandWrapper, ctx inte
}
ui.Say(fmt.Sprintf("Executing command: %s", command))
comm := &sl.Communicator{
ExecuteCommand: []string{"sh", "-c", command},
}
comm := &shell_local.Communicator{}
cmd := &packer.RemoteCmd{Command: command}
if err := cmd.StartWithUi(comm, ui); err != nil {
return fmt.Errorf("Error executing command: %s", err)
+52 -4
View File
@@ -2,10 +2,11 @@ package chroot
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
@@ -23,7 +24,7 @@ type StepAttachVolume struct {
volumeId string
}
func (s *StepAttachVolume) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepAttachVolume) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
device := state.Get("device").(string)
instance := state.Get("instance").(*ec2.Instance)
@@ -51,7 +52,35 @@ func (s *StepAttachVolume) Run(ctx context.Context, state multistep.StateBag) mu
s.volumeId = volumeId
// Wait for the volume to become attached
err = awscommon.WaitUntilVolumeAttached(ctx, ec2conn, s.volumeId)
stateChange := awscommon.StateChangeConf{
Pending: []string{"attaching"},
StepState: state,
Target: "attached",
Refresh: func() (interface{}, string, error) {
attempts := 0
for attempts < 30 {
resp, err := ec2conn.DescribeVolumes(&ec2.DescribeVolumesInput{VolumeIds: []*string{&volumeId}})
if err != nil {
return nil, "", err
}
if len(resp.Volumes[0].Attachments) > 0 {
a := resp.Volumes[0].Attachments[0]
return a, *a.State, nil
}
// When Attachment on volume is not present sleep for 2s and retry
attempts += 1
ui.Say(fmt.Sprintf(
"Volume %s show no attachments. Attempt %d/30. Sleeping for 2s and will retry.",
volumeId, attempts))
time.Sleep(2 * time.Second)
}
// Attachment on volume is not present after all attempts
return nil, "", errors.New("No attachments on volume.")
},
}
_, err = awscommon.WaitForState(&stateChange)
if err != nil {
err := fmt.Errorf("Error waiting for volume: %s", err)
state.Put("error", err)
@@ -87,7 +116,26 @@ func (s *StepAttachVolume) CleanupFunc(state multistep.StateBag) error {
s.attached = false
// Wait for the volume to detach
err = awscommon.WaitUntilVolumeDetached(aws.BackgroundContext(), ec2conn, s.volumeId)
stateChange := awscommon.StateChangeConf{
Pending: []string{"attaching", "attached", "detaching"},
StepState: state,
Target: "detached",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeVolumes(&ec2.DescribeVolumesInput{VolumeIds: []*string{&s.volumeId}})
if err != nil {
return nil, "", err
}
v := resp.Volumes[0]
if len(v.Attachments) > 0 {
return v, *v.Attachments[0].State, nil
} else {
return v, "detached", nil
}
},
}
_, err = awscommon.WaitForState(&stateChange)
if err != nil {
return fmt.Errorf("Error waiting for volume: %s", err)
}
+34 -75
View File
@@ -2,7 +2,6 @@ package chroot
import (
"context"
"errors"
"fmt"
"log"
@@ -11,7 +10,6 @@ import (
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
)
// StepCreateVolume creates a new volume from the snapshot of the root
@@ -22,54 +20,21 @@ import (
type StepCreateVolume struct {
volumeId string
RootVolumeSize int64
RootVolumeType string
RootVolumeTags awscommon.TagMap
Ctx interpolate.Context
}
func (s *StepCreateVolume) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepCreateVolume) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
ui := state.Get("ui").(packer.Ui)
volTags, err := s.RootVolumeTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging volumes: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
// Collect tags for tagging on resource creation
var tagSpecs []*ec2.TagSpecification
if len(volTags) > 0 {
runVolTags := &ec2.TagSpecification{
ResourceType: aws.String("volume"),
Tags: volTags,
}
tagSpecs = append(tagSpecs, runVolTags)
}
var createVolume *ec2.CreateVolumeInput
if config.FromScratch {
rootVolumeType := ec2.VolumeTypeGp2
if s.RootVolumeType == "io1" {
err := errors.New("Cannot use io1 volume when building from scratch")
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
} else if s.RootVolumeType != "" {
rootVolumeType = s.RootVolumeType
}
createVolume = &ec2.CreateVolumeInput{
AvailabilityZone: instance.Placement.AvailabilityZone,
Size: aws.Int64(s.RootVolumeSize),
VolumeType: aws.String(rootVolumeType),
VolumeType: aws.String(ec2.VolumeTypeGp2),
}
} else {
// Determine the root device snapshot
image := state.Get("source_image").(*ec2.Image)
@@ -82,19 +47,28 @@ func (s *StepCreateVolume) Run(ctx context.Context, state multistep.StateBag) mu
}
}
ui.Say("Creating the root volume...")
createVolume, err = s.buildCreateVolumeInput(*instance.Placement.AvailabilityZone, rootDevice)
if err != nil {
if rootDevice == nil {
err := fmt.Errorf("Couldn't find root device!")
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ui.Say("Creating the root volume...")
vs := *rootDevice.Ebs.VolumeSize
if s.RootVolumeSize > *rootDevice.Ebs.VolumeSize {
vs = s.RootVolumeSize
}
createVolume = &ec2.CreateVolumeInput{
AvailabilityZone: instance.Placement.AvailabilityZone,
Size: aws.Int64(vs),
SnapshotId: rootDevice.Ebs.SnapshotId,
VolumeType: rootDevice.Ebs.VolumeType,
Iops: rootDevice.Ebs.Iops,
}
}
if len(tagSpecs) > 0 {
createVolume.SetTagSpecifications(tagSpecs)
volTags.Report(ui)
}
log.Printf("Create args: %+v", createVolume)
createVolumeResp, err := ec2conn.CreateVolume(createVolume)
@@ -110,7 +84,22 @@ func (s *StepCreateVolume) Run(ctx context.Context, state multistep.StateBag) mu
log.Printf("Volume ID: %s", s.volumeId)
// Wait for the volume to become ready
err = awscommon.WaitUntilVolumeAvailable(ctx, ec2conn, s.volumeId)
stateChange := awscommon.StateChangeConf{
Pending: []string{"creating"},
StepState: state,
Target: "available",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeVolumes(&ec2.DescribeVolumesInput{VolumeIds: []*string{&s.volumeId}})
if err != nil {
return nil, "", err
}
v := resp.Volumes[0]
return v, *v.State, nil
},
}
_, err = awscommon.WaitForState(&stateChange)
if err != nil {
err := fmt.Errorf("Error waiting for volume: %s", err)
state.Put("error", err)
@@ -136,33 +125,3 @@ func (s *StepCreateVolume) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error deleting EBS volume: %s", err))
}
}
func (s *StepCreateVolume) buildCreateVolumeInput(az string, rootDevice *ec2.BlockDeviceMapping) (*ec2.CreateVolumeInput, error) {
if rootDevice == nil {
return nil, fmt.Errorf("Couldn't find root device!")
}
createVolumeInput := &ec2.CreateVolumeInput{
AvailabilityZone: aws.String(az),
Size: rootDevice.Ebs.VolumeSize,
SnapshotId: rootDevice.Ebs.SnapshotId,
VolumeType: rootDevice.Ebs.VolumeType,
Iops: rootDevice.Ebs.Iops,
}
if s.RootVolumeSize > *rootDevice.Ebs.VolumeSize {
createVolumeInput.Size = aws.Int64(s.RootVolumeSize)
}
if s.RootVolumeType == "" || s.RootVolumeType == *rootDevice.Ebs.VolumeType {
return createVolumeInput, nil
}
if s.RootVolumeType == "io1" {
return nil, fmt.Errorf("Root volume type cannot be io1, because existing root volume type was %s", *rootDevice.Ebs.VolumeType)
}
createVolumeInput.VolumeType = aws.String(s.RootVolumeType)
// non io1 cannot set iops
createVolumeInput.Iops = nil
return createVolumeInput, nil
}
@@ -1,73 +0,0 @@
package chroot
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/stretchr/testify/assert"
"testing"
)
func buildTestRootDevice() *ec2.BlockDeviceMapping {
return &ec2.BlockDeviceMapping{
Ebs: &ec2.EbsBlockDevice{
VolumeSize: aws.Int64(10),
SnapshotId: aws.String("snap-1234"),
VolumeType: aws.String("gp2"),
},
}
}
func TestCreateVolume_Default(t *testing.T) {
stepCreateVolume := new(StepCreateVolume)
_, err := stepCreateVolume.buildCreateVolumeInput("test-az", buildTestRootDevice())
assert.NoError(t, err)
}
func TestCreateVolume_Shrink(t *testing.T) {
stepCreateVolume := StepCreateVolume{RootVolumeSize: 1}
testRootDevice := buildTestRootDevice()
ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
assert.NoError(t, err)
// Ensure that the new value is equal to the size of the old root device
assert.Equal(t, *ret.Size, *testRootDevice.Ebs.VolumeSize)
}
func TestCreateVolume_Expand(t *testing.T) {
stepCreateVolume := StepCreateVolume{RootVolumeSize: 25}
testRootDevice := buildTestRootDevice()
ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
assert.NoError(t, err)
// Ensure that the new value is equal to the size of the value passed in
assert.Equal(t, *ret.Size, stepCreateVolume.RootVolumeSize)
}
func TestCreateVolume_io1_to_io1(t *testing.T) {
stepCreateVolume := StepCreateVolume{RootVolumeType: "io1"}
testRootDevice := buildTestRootDevice()
testRootDevice.Ebs.VolumeType = aws.String("io1")
testRootDevice.Ebs.Iops = aws.Int64(1000)
ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
assert.NoError(t, err)
assert.Equal(t, *ret.VolumeType, stepCreateVolume.RootVolumeType)
assert.Equal(t, *ret.Iops, *testRootDevice.Ebs.Iops)
}
func TestCreateVolume_io1_to_gp2(t *testing.T) {
stepCreateVolume := StepCreateVolume{RootVolumeType: "gp2"}
testRootDevice := buildTestRootDevice()
testRootDevice.Ebs.VolumeType = aws.String("io1")
testRootDevice.Ebs.Iops = aws.Int64(1000)
ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
assert.NoError(t, err)
assert.Equal(t, *ret.VolumeType, stepCreateVolume.RootVolumeType)
assert.Nil(t, ret.Iops)
}
func TestCreateVolume_gp2_to_io1(t *testing.T) {
stepCreateVolume := StepCreateVolume{RootVolumeType: "io1"}
testRootDevice := buildTestRootDevice()
_, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
assert.Error(t, err)
}
+4 -10
View File
@@ -26,7 +26,7 @@ type mountPathData struct {
// mount_device_cleanup CleanupFunc - To perform early cleanup
type StepMountDevice struct {
MountOptions []string
MountPartition string
MountPartition int
mountPath string
}
@@ -35,10 +35,6 @@ func (s *StepMountDevice) Run(_ context.Context, state multistep.StateBag) multi
config := state.Get("config").(*Config)
ui := state.Get("ui").(packer.Ui)
device := state.Get("device").(string)
if config.NVMEDevicePath != "" {
// customizable device path for mounting NVME block devices on c5 and m5 HVM
device = config.NVMEDevicePath
}
wrappedCommand := state.Get("wrappedCommand").(CommandWrapper)
var virtualizationType string
@@ -51,7 +47,6 @@ func (s *StepMountDevice) Run(_ context.Context, state multistep.StateBag) multi
}
ctx := config.ctx
ctx.Data = &mountPathData{Device: filepath.Base(device)}
mountPath, err := interpolate.Render(config.MountPath, &ctx)
@@ -80,9 +75,8 @@ func (s *StepMountDevice) Run(_ context.Context, state multistep.StateBag) multi
}
deviceMount := device
if virtualizationType == "hvm" && s.MountPartition != "0" {
deviceMount = fmt.Sprintf("%s%s", device, s.MountPartition)
if virtualizationType == "hvm" {
deviceMount = fmt.Sprintf("%s%d", device, s.MountPartition)
}
state.Put("deviceMount", deviceMount)
@@ -103,7 +97,7 @@ func (s *StepMountDevice) Run(_ context.Context, state multistep.StateBag) multi
ui.Error(err.Error())
return multistep.ActionHalt
}
log.Printf("[DEBUG] (step mount) mount command is %s", mountCommand)
cmd := ShellCommand(mountCommand)
cmd.Stderr = stderr
if err := cmd.Run(); err != nil {
+12 -4
View File
@@ -14,11 +14,11 @@ import (
// StepRegisterAMI creates the AMI.
type StepRegisterAMI struct {
RootVolumeSize int64
EnableAMIENASupport *bool
EnableAMIENASupport bool
EnableAMISriovNetSupport bool
}
func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepRegisterAMI) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
snapshotId := state.Get("snapshot_id").(string)
@@ -83,7 +83,7 @@ func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) mul
// As of February 2017, this applies to C3, C4, D2, I2, R3, and M4 (excluding m4.16xlarge)
registerOpts.SriovNetSupport = aws.String("simple")
}
if s.EnableAMIENASupport != nil && *s.EnableAMIENASupport {
if s.EnableAMIENASupport {
// Set EnaSupport to true
// As of February 2017, this applies to C5, I3, P2, R4, X1, and m4.16xlarge
registerOpts.EnaSupport = aws.Bool(true)
@@ -102,8 +102,16 @@ func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) mul
amis[*ec2conn.Config.Region] = *registerResp.ImageId
state.Put("amis", amis)
// Wait for the image to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *registerResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *registerResp.ImageId); err != nil {
if _, err := awscommon.WaitForState(&stateChange); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
+22 -2
View File
@@ -2,6 +2,7 @@ package chroot
import (
"context"
"errors"
"fmt"
"time"
@@ -19,7 +20,7 @@ type StepSnapshot struct {
snapshotId string
}
func (s *StepSnapshot) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepSnapshot) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
volumeId := state.Get("volume_id").(string)
@@ -43,7 +44,26 @@ func (s *StepSnapshot) Run(ctx context.Context, state multistep.StateBag) multis
ui.Message(fmt.Sprintf("Snapshot ID: %s", s.snapshotId))
// Wait for the snapshot to be ready
err = awscommon.WaitUntilSnapshotDone(ctx, ec2conn, s.snapshotId)
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
StepState: state,
Target: "completed",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeSnapshots(&ec2.DescribeSnapshotsInput{SnapshotIds: []*string{&s.snapshotId}})
if err != nil {
return nil, "", err
}
if len(resp.Snapshots) == 0 {
return nil, "", errors.New("No snapshots found.")
}
s := resp.Snapshots[0]
return s, *s.State, nil
},
}
_, err = awscommon.WaitForState(&stateChange)
if err != nil {
err := fmt.Errorf("Error waiting for snapshot: %s", err)
state.Put("error", err)
+57 -64
View File
@@ -1,39 +1,32 @@
package common
import (
"crypto/tls"
"fmt"
"log"
"net/http"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
cleanhttp "github.com/hashicorp/go-cleanhttp"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/go-cleanhttp"
"github.com/hashicorp/packer/template/interpolate"
)
// AccessConfig is for common configuration related to AWS access
type AccessConfig struct {
AccessKey string `mapstructure:"access_key"`
CustomEndpointEc2 string `mapstructure:"custom_endpoint_ec2"`
DecodeAuthZMessages bool `mapstructure:"decode_authorization_messages"`
InsecureSkipTLSVerify bool `mapstructure:"insecure_skip_tls_verify"`
MFACode string `mapstructure:"mfa_code"`
ProfileName string `mapstructure:"profile"`
RawRegion string `mapstructure:"region"`
SecretKey string `mapstructure:"secret_key"`
SkipValidation bool `mapstructure:"skip_region_validation"`
SkipMetadataApiCheck bool `mapstructure:"skip_metadata_api_check"`
Token string `mapstructure:"token"`
session *session.Session
getEC2Connection func() ec2iface.EC2API
AccessKey string `mapstructure:"access_key"`
CustomEndpointEc2 string `mapstructure:"custom_endpoint_ec2"`
MFACode string `mapstructure:"mfa_code"`
ProfileName string `mapstructure:"profile"`
RawRegion string `mapstructure:"region"`
SecretKey string `mapstructure:"secret_key"`
SkipValidation bool `mapstructure:"skip_region_validation"`
SkipMetadataApiCheck bool `mapstructure:"skip_metadata_api_check"`
Token string `mapstructure:"token"`
session *session.Session
}
// Config returns a valid aws.Config object for access to AWS services, or
@@ -44,6 +37,7 @@ func (c *AccessConfig) Session() (*session.Session, error) {
}
config := aws.NewConfig().WithCredentialsChainVerboseErrors(true)
staticCreds := credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, c.Token)
if _, err := staticCreds.Get(); err != credentials.ErrStaticCredentialsEmpty {
config.WithCredentials(staticCreds)
@@ -51,20 +45,14 @@ func (c *AccessConfig) Session() (*session.Session, error) {
if c.RawRegion != "" {
config = config.WithRegion(c.RawRegion)
} else if region := c.metadataRegion(); region != "" {
config = config.WithRegion(region)
}
if c.CustomEndpointEc2 != "" {
config = config.WithEndpoint(c.CustomEndpointEc2)
}
if c.InsecureSkipTLSVerify {
config := config.WithHTTPClient(cleanhttp.DefaultClient())
transport := config.HTTPClient.Transport.(*http.Transport)
transport.TLSClientConfig = &tls.Config{
InsecureSkipVerify: true,
}
}
opts := session.Options{
SharedConfigState: session.SharedConfigEnable,
Config: *config,
@@ -80,30 +68,26 @@ func (c *AccessConfig) Session() (*session.Session, error) {
}
}
sess, err := session.NewSessionWithOptions(opts)
if err != nil {
if sess, err := session.NewSessionWithOptions(opts); err != nil {
return nil, err
}
log.Printf("Found region %s", *sess.Config.Region)
c.session = sess
} else if *sess.Config.Region == "" {
return nil, fmt.Errorf("Could not find AWS region, make sure it's set.")
} else {
log.Printf("Found region %s", *sess.Config.Region)
c.session = sess
cp, err := c.session.Config.Credentials.Get()
if err != nil {
if awsErr, ok := err.(awserr.Error); ok && awsErr.Code() == "NoCredentialProviders" {
return nil, fmt.Errorf("No valid credential sources found for AWS Builder. " +
"Please see https://www.packer.io/docs/builders/amazon.html#specifying-amazon-credentials " +
"for more information on providing credentials for the AWS Builder.")
} else {
return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err)
cp, err := c.session.Config.Credentials.Get()
if err != nil {
if awsErr, ok := err.(awserr.Error); ok && awsErr.Code() == "NoCredentialProviders" {
return nil, fmt.Errorf("No valid credential sources found for AWS Builder. " +
"Please see https://www.packer.io/docs/builders/amazon.html#specifying-amazon-credentials " +
"for more information on providing credentials for the AWS Builder.")
} else {
return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err)
}
}
log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName)
}
log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName)
if c.DecodeAuthZMessages {
DecodeAuthZMessages(c.session)
}
LogEnvOverrideWarnings()
return c.session, nil
}
@@ -122,6 +106,25 @@ func (c *AccessConfig) IsChinaCloud() bool {
return strings.HasPrefix(c.SessionRegion(), "cn-")
}
// metadataRegion returns the region from the metadata service
func (c *AccessConfig) metadataRegion() string {
client := cleanhttp.DefaultClient()
// Keep the default timeout (100ms) low as we don't want to wait in non-EC2 environments
client.Timeout = 100 * time.Millisecond
ec2meta := ec2metadata.New(session.New(), &aws.Config{
HTTPClient: client,
})
region, err := ec2meta.Region()
if err != nil {
log.Println("Error getting region from metadata service, "+
"probably because we're not running on AWS.", err)
return ""
}
return region
}
func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
var errs []error
@@ -135,21 +138,11 @@ func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
fmt.Errorf("`access_key` and `secret_key` must both be either set or not set."))
}
if c.RawRegion != "" && !c.SkipValidation {
if valid := ValidateRegion(c.RawRegion); !valid {
errs = append(errs, fmt.Errorf("Unknown region: %s", c.RawRegion))
}
}
return errs
}
func (c *AccessConfig) NewEC2Connection() (ec2iface.EC2API, error) {
if c.getEC2Connection != nil {
return c.getEC2Connection(), nil
}
sess, err := c.Session()
if err != nil {
return nil, err
}
ec2conn := ec2.New(sess, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
})
return ec2conn, nil
}
+24 -34
View File
@@ -5,51 +5,41 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
)
func testAccessConfig() *AccessConfig {
return &AccessConfig{
getEC2Connection: func() ec2iface.EC2API {
return &mockEC2Client{}
},
}
return &AccessConfig{}
}
func TestAccessConfigPrepare_Region(t *testing.T) {
c := testAccessConfig()
c.RawRegion = "us-east-12"
err := c.ValidateRegion(c.RawRegion)
if err == nil {
t.Fatalf("should have region validation err: %s", c.RawRegion)
}
c.RawRegion = "us-east-1"
err = c.ValidateRegion(c.RawRegion)
if err != nil {
t.Fatalf("shouldn't have region validation err: %s", c.RawRegion)
}
c.RawRegion = "custom"
err = c.ValidateRegion(c.RawRegion)
if err == nil {
t.Fatalf("should have region validation err: %s", c.RawRegion)
}
c.RawRegion = "custom"
c.SkipValidation = true
// testing whole prepare func here; this is checking that validation is
// skipped, so we don't need a mock connection
if err := c.Prepare(nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
c.SkipValidation = false
c.RawRegion = ""
if err := c.Prepare(nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
c.RawRegion = "us-east-12"
if err := c.Prepare(nil); err == nil {
t.Fatal("should have error")
}
c.RawRegion = "us-east-1"
if err := c.Prepare(nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
c.RawRegion = "custom"
if err := c.Prepare(nil); err == nil {
t.Fatalf("should have err")
}
c.RawRegion = "custom"
c.SkipValidation = true
if err := c.Prepare(nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
c.SkipValidation = false
}
func TestAccessConfigPrepare_RegionRestricted(t *testing.T) {
+39 -75
View File
@@ -3,7 +3,6 @@ package common
import (
"fmt"
"log"
"regexp"
"github.com/hashicorp/packer/template/interpolate"
)
@@ -19,7 +18,7 @@ type AMIConfig struct {
AMIRegions []string `mapstructure:"ami_regions"`
AMISkipRegionValidation bool `mapstructure:"skip_region_validation"`
AMITags TagMap `mapstructure:"tags"`
AMIENASupport *bool `mapstructure:"ena_support"`
AMIENASupport bool `mapstructure:"ena_support"`
AMISriovNetSupport bool `mapstructure:"sriov_support"`
AMIForceDeregister bool `mapstructure:"force_deregister"`
AMIForceDeleteSnapshot bool `mapstructure:"force_delete_snapshot"`
@@ -57,29 +56,49 @@ func (c *AMIConfig) Prepare(accessConfig *AccessConfig, ctx *interpolate.Context
}
}
errs = append(errs, c.prepareRegions(accessConfig)...)
if len(c.AMIRegions) > 0 {
regionSet := make(map[string]struct{})
regions := make([]string, 0, len(c.AMIRegions))
for _, region := range c.AMIRegions {
// If we already saw the region, then don't look again
if _, ok := regionSet[region]; ok {
continue
}
// Mark that we saw the region
regionSet[region] = struct{}{}
if !c.AMISkipRegionValidation {
// Verify the region is real
if valid := ValidateRegion(region); !valid {
errs = append(errs, fmt.Errorf("Unknown region: %s", region))
}
}
// Make sure that if we have region_kms_key_ids defined,
// the regions in ami_regions are also in region_kms_key_ids
if len(c.AMIRegionKMSKeyIDs) > 0 {
if _, ok := c.AMIRegionKMSKeyIDs[region]; !ok {
errs = append(errs, fmt.Errorf("Region %s is in ami_regions but not in region_kms_key_ids", region))
}
}
if (accessConfig != nil) && (region == accessConfig.RawRegion) {
// make sure we don't try to copy to the region we originally
// create the AMI in.
log.Printf("Cannot copy AMI to AWS session region '%s', deleting it from `ami_regions`.", region)
continue
}
regions = append(regions, region)
}
c.AMIRegions = regions
}
if len(c.AMIUsers) > 0 && c.AMIEncryptBootVolume {
errs = append(errs, fmt.Errorf("Cannot share AMI with encrypted boot volume"))
}
var kmsKeys []string
if len(c.AMIKmsKeyId) > 0 {
kmsKeys = append(kmsKeys, c.AMIKmsKeyId)
}
if len(c.AMIRegionKMSKeyIDs) > 0 {
for _, kmsKey := range c.AMIRegionKMSKeyIDs {
if len(kmsKey) == 0 {
kmsKeys = append(kmsKeys, c.AMIKmsKeyId)
}
}
}
for _, kmsKey := range kmsKeys {
if !validateKmsKey(kmsKey) {
errs = append(errs, fmt.Errorf("%s is not a valid KMS Key Id.", kmsKey))
}
}
if len(c.SnapshotUsers) > 0 {
if len(c.AMIKmsKeyId) == 0 && c.AMIEncryptBootVolume {
errs = append(errs, fmt.Errorf("Cannot share snapshot encrypted with default KMS key"))
@@ -111,58 +130,3 @@ func (c *AMIConfig) Prepare(accessConfig *AccessConfig, ctx *interpolate.Context
return nil
}
func (c *AMIConfig) prepareRegions(accessConfig *AccessConfig) (errs []error) {
if len(c.AMIRegions) > 0 {
regionSet := make(map[string]struct{})
regions := make([]string, 0, len(c.AMIRegions))
for _, region := range c.AMIRegions {
// If we already saw the region, then don't look again
if _, ok := regionSet[region]; ok {
continue
}
// Mark that we saw the region
regionSet[region] = struct{}{}
// Make sure that if we have region_kms_key_ids defined,
// the regions in ami_regions are also in region_kms_key_ids
if len(c.AMIRegionKMSKeyIDs) > 0 {
if _, ok := c.AMIRegionKMSKeyIDs[region]; !ok {
errs = append(errs, fmt.Errorf("Region %s is in ami_regions but not in region_kms_key_ids", region))
}
}
if (accessConfig != nil) && (region == accessConfig.RawRegion) {
// make sure we don't try to copy to the region we originally
// create the AMI in.
log.Printf("Cannot copy AMI to AWS session region '%s', deleting it from `ami_regions`.", region)
continue
}
regions = append(regions, region)
}
c.AMIRegions = regions
}
return errs
}
// See https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CopyImage.html
func validateKmsKey(kmsKey string) (valid bool) {
kmsKeyIdPattern := `[a-f0-9-]+$`
aliasPattern := `alias/[a-zA-Z0-9:/_-]+$`
kmsArnStartPattern := `^arn:aws:kms:([a-z]{2}-(gov-)?[a-z]+-\d{1})?:(\d{12}):`
if regexp.MustCompile(fmt.Sprintf("^%s", kmsKeyIdPattern)).MatchString(kmsKey) {
return true
}
if regexp.MustCompile(fmt.Sprintf("^%s", aliasPattern)).MatchString(kmsKey) {
return true
}
if regexp.MustCompile(fmt.Sprintf("%skey/%s", kmsArnStartPattern, kmsKeyIdPattern)).MatchString(kmsKey) {
return true
}
if regexp.MustCompile(fmt.Sprintf("%s%s", kmsArnStartPattern, aliasPattern)).MatchString(kmsKey) {
return true
}
return false
}
+35 -97
View File
@@ -1,13 +1,8 @@
package common
import (
"fmt"
"reflect"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
)
func testAMIConfig() *AMIConfig {
@@ -17,62 +12,43 @@ func testAMIConfig() *AMIConfig {
}
func getFakeAccessConfig(region string) *AccessConfig {
c := testAccessConfig()
c.RawRegion = region
return c
return &AccessConfig{
RawRegion: region,
}
}
func TestAMIConfigPrepare_name(t *testing.T) {
c := testAMIConfig()
accessConf := testAccessConfig()
if err := c.Prepare(accessConf, nil); err != nil {
if err := c.Prepare(nil, nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
c.AMIName = ""
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have error")
}
}
type mockEC2Client struct {
ec2iface.EC2API
}
func (m *mockEC2Client) DescribeRegions(*ec2.DescribeRegionsInput) (*ec2.DescribeRegionsOutput, error) {
return &ec2.DescribeRegionsOutput{
Regions: []*ec2.Region{
{RegionName: aws.String("us-east-1")},
{RegionName: aws.String("us-east-2")},
{RegionName: aws.String("us-west-1")},
},
}, nil
}
func TestAMIConfigPrepare_regions(t *testing.T) {
c := testAMIConfig()
c.AMIRegions = nil
var errs []error
var err error
accessConf := testAccessConfig()
mockConn := &mockEC2Client{}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
t.Fatalf("shouldn't have err: %#v", errs)
if err := c.Prepare(nil, nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
c.AMIRegions, err = listEC2Regions(mockConn)
if err != nil {
t.Fatalf("shouldn't have err: %s", err.Error())
c.AMIRegions = listEC2Regions()
if err := c.Prepare(nil, nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
t.Fatalf("shouldn't have err: %#v", errs)
c.AMIRegions = []string{"foo"}
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have error")
}
errs = errs[:0]
c.AMIRegions = []string{"us-east-1", "us-west-1", "us-east-1"}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
t.Fatalf("bad: %s", errs[0])
if err := c.Prepare(nil, nil); err != nil {
t.Fatalf("bad: %s", err)
}
expected := []string{"us-east-1", "us-west-1"}
@@ -81,9 +57,11 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
}
c.AMIRegions = []string{"custom"}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
c.AMISkipRegionValidation = true
if err := c.Prepare(nil, nil); err != nil {
t.Fatal("shouldn't have error")
}
c.AMISkipRegionValidation = false
c.AMIRegions = []string{"us-east-1", "us-east-2", "us-west-1"}
c.AMIRegionKMSKeyIDs = map[string]string{
@@ -91,8 +69,8 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
"us-west-1": "789-012-3456",
"us-east-2": "456-789-0123",
}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
t.Fatal(fmt.Sprintf("shouldn't have error: %s", errs[0]))
if err := c.Prepare(nil, nil); err != nil {
t.Fatal("shouldn't have error")
}
c.AMIRegions = []string{"us-east-1", "us-east-2", "us-west-1"}
@@ -101,7 +79,7 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
"us-west-1": "789-012-3456",
"us-east-2": "",
}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
if err := c.Prepare(nil, nil); err != nil {
t.Fatal("should have passed; we are able to use default KMS key if not sharing")
}
@@ -112,7 +90,7 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
"us-west-1": "789-012-3456",
"us-east-2": "",
}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have an error b/c can't use default KMS key if sharing")
}
@@ -122,7 +100,7 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
"us-west-1": "789-012-3456",
"us-east-2": "456-789-0123",
}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have error b/c theres a region in the key map that isn't in ami_regions")
}
@@ -131,8 +109,7 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
"us-east-1": "123-456-7890",
"us-west-1": "789-012-3456",
}
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have error b/c theres a region in in ami_regions that isn't in the key map")
}
@@ -144,15 +121,15 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
"us-east-1": "123-456-7890",
"us-west-1": "",
}
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have error b/c theres a region in in ami_regions that isn't in the key map")
}
// allow rawregion to exist in ami_regions list.
accessConf = getFakeAccessConfig("us-east-1")
accessConf := getFakeAccessConfig("us-east-1")
c.AMIRegions = []string{"us-east-1", "us-west-1", "us-east-2"}
c.AMIRegionKMSKeyIDs = nil
if errs = c.prepareRegions(accessConf); len(errs) > 0 {
if err := c.Prepare(accessConf, nil); err != nil {
t.Fatal("should allow user to have the raw region in ami_regions")
}
@@ -163,61 +140,22 @@ func TestAMIConfigPrepare_Share_EncryptedBoot(t *testing.T) {
c.AMIUsers = []string{"testAccountID"}
c.AMIEncryptBootVolume = true
accessConf := testAccessConfig()
c.AMIKmsKeyId = ""
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("shouldn't be able to share ami with encrypted boot volume")
}
c.AMIKmsKeyId = "89c3fb9a-de87-4f2a-aedc-fddc5138193c"
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("shouldn't be able to share ami with encrypted boot volume")
}
}
func TestAMIConfigPrepare_ValidateKmsKey(t *testing.T) {
c := testAMIConfig()
c.AMIEncryptBootVolume = true
accessConf := testAccessConfig()
validCases := []string{
"abcd1234-e567-890f-a12b-a123b4cd56ef",
"alias/foo/bar",
"arn:aws:kms:us-east-1:012345678910:key/abcd1234-a123-456a-a12b-a123b4cd56ef",
"arn:aws:kms:us-east-1:012345678910:alias/foo/bar",
}
for _, validCase := range validCases {
c.AMIKmsKeyId = validCase
if err := c.Prepare(accessConf, nil); err != nil {
t.Fatalf("%s should not have failed KMS key validation", validCase)
}
}
invalidCases := []string{
"ABCD1234-e567-890f-a12b-a123b4cd56ef",
"ghij1234-e567-890f-a12b-a123b4cd56ef",
"ghij1234+e567_890f-a12b-a123b4cd56ef",
"foo/bar",
"arn:aws:kms:us-east-1:012345678910:foo/bar",
}
for _, invalidCase := range invalidCases {
c.AMIKmsKeyId = invalidCase
if err := c.Prepare(accessConf, nil); err == nil {
t.Fatalf("%s should have failed KMS key validation", invalidCase)
}
}
}
func TestAMINameValidation(t *testing.T) {
c := testAMIConfig()
accessConf := testAccessConfig()
c.AMIName = "aa"
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("shouldn't be able to have an ami name with less than 3 characters")
}
@@ -226,22 +164,22 @@ func TestAMINameValidation(t *testing.T) {
longAmiName += "a"
}
c.AMIName = longAmiName
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("shouldn't be able to have an ami name with great than 128 characters")
}
c.AMIName = "+aaa"
if err := c.Prepare(accessConf, nil); err == nil {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("shouldn't be able to have an ami name with invalid characters")
}
c.AMIName = "fooBAR1()[] ./-'@_"
if err := c.Prepare(accessConf, nil); err != nil {
if err := c.Prepare(nil, nil); err != nil {
t.Fatal("should be able to use all of the allowed AMI characters")
}
c.AMIName = `xyz-base-2017-04-05-1934`
if err := c.Prepare(accessConf, nil); err != nil {
if err := c.Prepare(nil, nil); err != nil {
t.Fatalf("expected `xyz-base-2017-04-05-1934` to pass validation.")
}
+1 -3
View File
@@ -9,7 +9,6 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/packer"
)
@@ -71,8 +70,7 @@ func (a *Artifact) Destroy() error {
log.Printf("Deregistering image ID (%s) from region (%s)", imageId, region)
regionConn := ec2.New(a.Session, &aws.Config{
Region: aws.String(region),
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
Region: aws.String(region),
})
// Get image metadata
-4
View File
@@ -88,10 +88,6 @@ func buildBlockDevices(b []BlockDevice) []*ec2.BlockDeviceMapping {
}
func (b *BlockDevice) Prepare(ctx *interpolate.Context) error {
if b.DeviceName == "" {
return fmt.Errorf("The `device_name` must be specified " +
"for every device in the block device mapping.")
}
// Warn that encrypted must be true when setting kms_key_id
if b.KmsKeyId != "" && b.Encrypted == false {
return fmt.Errorf("The device %v, must also have `encrypted: "+
-17
View File
@@ -1,17 +0,0 @@
package common
import (
"github.com/aws/aws-sdk-go/service/ec2"
)
// Build a slice of EC2 (AMI/Subnet/VPC) filter options from the filters provided.
func buildEc2Filters(input map[*string]*string) []*ec2.Filter {
var filters []*ec2.Filter
for k, v := range input {
filters = append(filters, &ec2.Filter{
Name: k,
Values: []*string{v},
})
}
return filters
}
-58
View File
@@ -1,58 +0,0 @@
package common
import (
"fmt"
"log"
"regexp"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/sts"
)
var encodedFailureMessagePattern = regexp.MustCompile(`(?i)(.*) Encoded authorization failure message: ([\w-]+) ?( .*)?`)
type stsDecoder interface {
DecodeAuthorizationMessage(input *sts.DecodeAuthorizationMessageInput) (*sts.DecodeAuthorizationMessageOutput, error)
}
// decodeError replaces encoded authorization messages with the
// decoded results
func decodeAWSError(decoder stsDecoder, err error) error {
groups := encodedFailureMessagePattern.FindStringSubmatch(err.Error())
if groups != nil && len(groups) > 1 {
result, decodeErr := decoder.DecodeAuthorizationMessage(&sts.DecodeAuthorizationMessageInput{
EncodedMessage: aws.String(groups[2]),
})
if decodeErr == nil {
msg := aws.StringValue(result.DecodedMessage)
return fmt.Errorf("%s Authorization failure message: '%s'%s", groups[1], msg, groups[3])
}
log.Printf("[WARN] Attempted to decode authorization message, but received: %v", decodeErr)
}
return err
}
// DecodeAuthZMessages enables automatic decoding of any
// encoded authorization messages
func DecodeAuthZMessages(sess *session.Session) {
azd := &authZMessageDecoder{
Decoder: sts.New(sess),
}
sess.Handlers.UnmarshalError.AfterEachFn = azd.afterEachFn
}
type authZMessageDecoder struct {
Decoder stsDecoder
}
func (a *authZMessageDecoder) afterEachFn(item request.HandlerListRunItem) bool {
if err, ok := item.Request.Error.(awserr.Error); ok && err.Code() == "UnauthorizedOperation" {
item.Request.Error = decodeAWSError(a.Decoder, err)
}
return true
}
-70
View File
@@ -1,70 +0,0 @@
package common
import (
"fmt"
"strings"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/sts"
"github.com/aws/aws-sdk-go/aws/awserr"
)
type mockSTS struct {
}
func (m *mockSTS) DecodeAuthorizationMessage(input *sts.DecodeAuthorizationMessageInput) (*sts.DecodeAuthorizationMessageOutput, error) {
return &sts.DecodeAuthorizationMessageOutput{
DecodedMessage: aws.String(`{
"allowed": false,
"explicitDeny": true,
"matchedStatements": {}
}`),
}, nil
}
func TestErrorsParsing_RequestFailure(t *testing.T) {
ae := awserr.New("UnauthorizedOperation",
`You are not authorized to perform this operation. Encoded authorization failure message: D9Q7oicjOMr9l2CC-NPP1FiZXK9Ijia1k-3l0siBFCcrK3oSuMFMkBIO5TNj0HdXE-WfwnAcdycFOohfKroNO6toPJEns8RFVfy_M_IjNGmrEFJ6E62pnmBW0OLrMsXxR9FQE4gB4gJzSM0AD6cV6S3FOfqYzWBRX-sQdOT4HryGkFNRoFBr9Xbp-tRwiadwkbdHdfnV9fbRkXmnwCdULml16NBSofC4ZPepLMKmIB5rKjwk-m179UUh2XA-J5no0si6XcRo5GbHQB5QfCIwSHL4vsro2wLZUd16-8OWKyr3tVlTbQe0ERZskqRqRQ5E28QuiBCVV6XstUyo-T4lBSr75Fgnyr3wCO-dS3b_5Ns3WzA2JD4E2AJOAStXIU8IH5YuKkAg7C-dJMuBMPpmKCBEXhNoHDwCyOo5PsV3xMlc0jSb0qYGpfst_TDDtejcZfn7NssUjxVq9qkdH-OXz2gPoQB-hX8ycmZCL5UZwKc3TCLUr7TGnudHjmnMrE9cUo-yTCWfyHPLprhiYhTCKW18EikJ0O1EKI3FJ_b4F19_jFBPARjSwQc7Ut6MNCVzrPdZGYSF6acj5gPaxdy9uSkVQwWXK7Pd5MFP7EBDE1_DgYbzodgwDO2PXeVFUbSLBHKWo_ebZS9ZX2nYPcGss_sYaly0ZVSIJXp7G58B5BoFVhvVH6jYnF9XiAOjMltuP_ycu1pQP1lki500RY3baLvfeYeAsB38XZHKEgWZzq7Fei-uh89q0cjJTmlVyrfRU3q6`,
fmt.Errorf("You can't do it!!"))
rf := awserr.NewRequestFailure(ae, 400, "abc-def-123-456")
result := decodeAWSError(&mockSTS{}, rf)
if result == nil {
t.Error("Expected resulting error")
}
if !strings.Contains(result.Error(), "Authorization failure message:") {
t.Error("Expected authorization failure message")
}
}
func TestErrorsParsing_NonAuthorizationFailure(t *testing.T) {
ae := awserr.New("BadRequest",
`You did something wrong. Try again`,
fmt.Errorf("Request was no good."))
rf := awserr.NewRequestFailure(ae, 400, "abc-def-123-456")
result := decodeAWSError(&mockSTS{}, rf)
if result == nil {
t.Error("Expected resulting error")
}
if result != rf {
t.Error("Expected original error to be returned unchanged")
}
}
func TestErrorsParsing_NonAWSError(t *testing.T) {
err := fmt.Errorf("Random error occurred")
result := decodeAWSError(&mockSTS{}, err)
if result == nil {
t.Error("Expected resulting error")
}
if result != err {
t.Error("Expected original error to be returned unchanged")
}
}
+27 -46
View File
@@ -1,56 +1,37 @@
package common
import (
"fmt"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
)
func listEC2Regions(ec2conn ec2iface.EC2API) ([]string, error) {
var regions []string
resultRegions, err := ec2conn.DescribeRegions(nil)
if err != nil {
return []string{}, err
// aws ec2 describe-regions --query 'Regions[].{Name:RegionName}' --output text | sed 's/\(.*\)/"\1",/' | sort
func listEC2Regions() []string {
return []string{
"ap-northeast-1",
"ap-northeast-2",
"ap-south-1",
"ap-southeast-1",
"ap-southeast-2",
"ca-central-1",
"eu-central-1",
"eu-west-1",
"eu-west-2",
"eu-west-3",
"sa-east-1",
"us-east-1",
"us-east-2",
"us-west-1",
"us-west-2",
// not part of autogenerated list
"us-gov-west-1",
"cn-north-1",
"cn-northwest-1",
}
for _, region := range resultRegions.Regions {
regions = append(regions, *region.RegionName)
}
return regions, nil
}
// ValidateRegion returns true if the supplied region is a valid AWS
// region and false if it's not.
func (c *AccessConfig) ValidateRegion(regions ...string) error {
ec2conn, err := c.NewEC2Connection()
if err != nil {
return err
}
validRegions, err := listEC2Regions(ec2conn)
if err != nil {
return err
}
var invalidRegions []string
for _, region := range regions {
if region == "" {
continue
}
found := false
for _, validRegion := range validRegions {
if region == validRegion {
found = true
break
}
}
if !found {
invalidRegions = append(invalidRegions, region)
func ValidateRegion(region string) bool {
for _, valid := range listEC2Regions() {
if region == valid {
return true
}
}
if len(invalidRegions) > 0 {
return fmt.Errorf("Invalid region(s): %v", invalidRegions)
}
return nil
return false
}
+42 -111
View File
@@ -1,11 +1,11 @@
package common
import (
"errors"
"fmt"
"net"
"os"
"regexp"
"strings"
"time"
"github.com/hashicorp/packer/common/uuid"
@@ -25,69 +25,35 @@ func (d *AmiFilterOptions) Empty() bool {
return len(d.Owners) == 0 && len(d.Filters) == 0
}
func (d *AmiFilterOptions) NoOwner() bool {
return len(d.Owners) == 0
}
type SubnetFilterOptions struct {
Filters map[*string]*string
MostFree bool `mapstructure:"most_free"`
Random bool `mapstructure:"random"`
}
func (d *SubnetFilterOptions) Empty() bool {
return len(d.Filters) == 0
}
type VpcFilterOptions struct {
Filters map[*string]*string
}
func (d *VpcFilterOptions) Empty() bool {
return len(d.Filters) == 0
}
type SecurityGroupFilterOptions struct {
Filters map[*string]*string
}
func (d *SecurityGroupFilterOptions) Empty() bool {
return len(d.Filters) == 0
}
// RunConfig contains configuration for running an instance from a source
// AMI and details on how to access that launched image.
type RunConfig struct {
AssociatePublicIpAddress bool `mapstructure:"associate_public_ip_address"`
AvailabilityZone string `mapstructure:"availability_zone"`
BlockDurationMinutes int64 `mapstructure:"block_duration_minutes"`
DisableStopInstance bool `mapstructure:"disable_stop_instance"`
EbsOptimized bool `mapstructure:"ebs_optimized"`
EnableT2Unlimited bool `mapstructure:"enable_t2_unlimited"`
IamInstanceProfile string `mapstructure:"iam_instance_profile"`
InstanceInitiatedShutdownBehavior string `mapstructure:"shutdown_behavior"`
InstanceType string `mapstructure:"instance_type"`
SecurityGroupFilter SecurityGroupFilterOptions `mapstructure:"security_group_filter"`
RunTags map[string]string `mapstructure:"run_tags"`
SecurityGroupId string `mapstructure:"security_group_id"`
SecurityGroupIds []string `mapstructure:"security_group_ids"`
SourceAmi string `mapstructure:"source_ami"`
SourceAmiFilter AmiFilterOptions `mapstructure:"source_ami_filter"`
SpotPrice string `mapstructure:"spot_price"`
SpotPriceAutoProduct string `mapstructure:"spot_price_auto_product"`
SpotTags map[string]string `mapstructure:"spot_tags"`
SubnetFilter SubnetFilterOptions `mapstructure:"subnet_filter"`
SubnetId string `mapstructure:"subnet_id"`
TemporaryKeyPairName string `mapstructure:"temporary_key_pair_name"`
TemporarySGSourceCidr string `mapstructure:"temporary_security_group_source_cidr"`
UserData string `mapstructure:"user_data"`
UserDataFile string `mapstructure:"user_data_file"`
VpcFilter VpcFilterOptions `mapstructure:"vpc_filter"`
VpcId string `mapstructure:"vpc_id"`
WindowsPasswordTimeout time.Duration `mapstructure:"windows_password_timeout"`
AssociatePublicIpAddress bool `mapstructure:"associate_public_ip_address"`
AvailabilityZone string `mapstructure:"availability_zone"`
EbsOptimized bool `mapstructure:"ebs_optimized"`
IamInstanceProfile string `mapstructure:"iam_instance_profile"`
InstanceType string `mapstructure:"instance_type"`
RunTags map[string]string `mapstructure:"run_tags"`
SourceAmi string `mapstructure:"source_ami"`
SourceAmiFilter AmiFilterOptions `mapstructure:"source_ami_filter"`
SpotPrice string `mapstructure:"spot_price"`
SpotPriceAutoProduct string `mapstructure:"spot_price_auto_product"`
DisableStopInstance bool `mapstructure:"disable_stop_instance"`
SecurityGroupId string `mapstructure:"security_group_id"`
SecurityGroupIds []string `mapstructure:"security_group_ids"`
TemporarySGSourceCidr string `mapstructure:"temporary_security_group_source_cidr"`
SubnetId string `mapstructure:"subnet_id"`
TemporaryKeyPairName string `mapstructure:"temporary_key_pair_name"`
UserData string `mapstructure:"user_data"`
UserDataFile string `mapstructure:"user_data_file"`
WindowsPasswordTimeout time.Duration `mapstructure:"windows_password_timeout"`
VpcId string `mapstructure:"vpc_id"`
InstanceInitiatedShutdownBehavior string `mapstructure:"shutdown_behavior"`
// Communicator settings
Comm communicator.Config `mapstructure:",squash"`
Comm communicator.Config `mapstructure:",squash"`
SSHKeyPairName string `mapstructure:"ssh_keypair_name"`
SSHInterface string `mapstructure:"ssh_interface"`
}
func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
@@ -95,10 +61,10 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
// ssh_private_key_file, then create a temporary one, but only if the
// temporary_key_pair_name has not been provided and we are not using
// ssh_password.
if c.Comm.SSHKeyPairName == "" && c.Comm.SSHTemporaryKeyPairName == "" &&
c.Comm.SSHPrivateKeyFile == "" && c.Comm.SSHPassword == "" {
if c.SSHKeyPairName == "" && c.TemporaryKeyPairName == "" &&
c.Comm.SSHPrivateKey == "" && c.Comm.SSHPassword == "" {
c.Comm.SSHTemporaryKeyPairName = fmt.Sprintf("packer_%s", uuid.TimeOrderedUUID())
c.TemporaryKeyPairName = fmt.Sprintf("packer_%s", uuid.TimeOrderedUUID())
}
if c.WindowsPasswordTimeout == 0 {
@@ -113,60 +79,37 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
errs := c.Comm.Prepare(ctx)
// Validating ssh_interface
if c.Comm.SSHInterface != "public_ip" &&
c.Comm.SSHInterface != "private_ip" &&
c.Comm.SSHInterface != "public_dns" &&
c.Comm.SSHInterface != "private_dns" &&
c.Comm.SSHInterface != "" {
errs = append(errs, fmt.Errorf("Unknown interface type: %s", c.Comm.SSHInterface))
if c.SSHInterface != "public_ip" &&
c.SSHInterface != "private_ip" &&
c.SSHInterface != "public_dns" &&
c.SSHInterface != "private_dns" &&
c.SSHInterface != "" {
errs = append(errs, errors.New(fmt.Sprintf("Unknown interface type: %s", c.SSHInterface)))
}
if c.Comm.SSHKeyPairName != "" {
if c.Comm.Type == "winrm" && c.Comm.WinRMPassword == "" && c.Comm.SSHPrivateKeyFile == "" {
errs = append(errs, fmt.Errorf("ssh_private_key_file must be provided to retrieve the winrm password when using ssh_keypair_name."))
} else if c.Comm.SSHPrivateKeyFile == "" && !c.Comm.SSHAgentAuth {
errs = append(errs, fmt.Errorf("ssh_private_key_file must be provided or ssh_agent_auth enabled when ssh_keypair_name is specified."))
if c.SSHKeyPairName != "" {
if c.Comm.Type == "winrm" && c.Comm.WinRMPassword == "" && c.Comm.SSHPrivateKey == "" {
errs = append(errs, errors.New("ssh_private_key_file must be provided to retrieve the winrm password when using ssh_keypair_name."))
} else if c.Comm.SSHPrivateKey == "" && !c.Comm.SSHAgentAuth {
errs = append(errs, errors.New("ssh_private_key_file must be provided or ssh_agent_auth enabled when ssh_keypair_name is specified."))
}
}
if c.SourceAmi == "" && c.SourceAmiFilter.Empty() {
errs = append(errs, fmt.Errorf("A source_ami or source_ami_filter must be specified"))
}
if c.SourceAmi == "" && c.SourceAmiFilter.NoOwner() {
errs = append(errs, fmt.Errorf("For security reasons, your source AMI filter must declare an owner."))
errs = append(errs, errors.New("A source_ami or source_ami_filter must be specified"))
}
if c.InstanceType == "" {
errs = append(errs, fmt.Errorf("An instance_type must be specified"))
}
if c.BlockDurationMinutes%60 != 0 {
errs = append(errs, fmt.Errorf(
"block_duration_minutes must be multiple of 60"))
errs = append(errs, errors.New("An instance_type must be specified"))
}
if c.SpotPrice == "auto" {
if c.SpotPriceAutoProduct == "" {
errs = append(errs, fmt.Errorf(
errs = append(errs, errors.New(
"spot_price_auto_product must be specified when spot_price is auto"))
}
}
if c.SpotPriceAutoProduct != "" {
if c.SpotPrice != "auto" {
errs = append(errs, fmt.Errorf(
"spot_price should be set to auto when spot_price_auto_product is specified"))
}
}
if c.SpotTags != nil {
if c.SpotPrice == "" || c.SpotPrice == "0" {
errs = append(errs, fmt.Errorf(
"spot_tags should not be set when not requesting a spot instance"))
}
}
if c.UserData != "" && c.UserDataFile != "" {
errs = append(errs, fmt.Errorf("Only one of user_data or user_data_file can be specified."))
} else if c.UserDataFile != "" {
@@ -198,18 +141,6 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
errs = append(errs, fmt.Errorf("shutdown_behavior only accepts 'stop' or 'terminate' values."))
}
if c.EnableT2Unlimited {
if c.SpotPrice != "" {
errs = append(errs, fmt.Errorf("Error: T2 Unlimited cannot be used in conjuction with Spot Instances"))
}
firstDotIndex := strings.Index(c.InstanceType, ".")
if firstDotIndex == -1 {
errs = append(errs, fmt.Errorf("Error determining main Instance Type from: %s", c.InstanceType))
} else if c.InstanceType[0:firstDotIndex] != "t2" {
errs = append(errs, fmt.Errorf("Error: T2 Unlimited enabled with a non-T2 Instance Type: %s", c.InstanceType))
}
}
return errs
}
+14 -67
View File
@@ -48,32 +48,22 @@ func TestRunConfigPrepare_InstanceType(t *testing.T) {
c := testConfig()
c.InstanceType = ""
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if an instance_type is not specified")
t.Fatalf("err: %s", err)
}
}
func TestRunConfigPrepare_SourceAmi(t *testing.T) {
c := testConfig()
c.SourceAmi = ""
if err := c.Prepare(nil); len(err) != 2 {
t.Fatalf("Should error if a source_ami (or source_ami_filter) is not specified")
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
}
}
func TestRunConfigPrepare_SourceAmiFilterBlank(t *testing.T) {
c := testConfigFilter()
if err := c.Prepare(nil); len(err) != 2 {
t.Fatalf("Should error if source_ami_filter is empty or not specified (and source_ami is not specified)")
}
}
func TestRunConfigPrepare_SourceAmiFilterOwnersBlank(t *testing.T) {
c := testConfigFilter()
filter_key := "name"
filter_value := "foo"
c.SourceAmiFilter = AmiFilterOptions{Filters: map[*string]*string{&filter_key: &filter_value}}
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if Owners is not specified)")
t.Fatalf("err: %s", err)
}
}
@@ -89,57 +79,16 @@ func TestRunConfigPrepare_SourceAmiFilterGood(t *testing.T) {
}
}
func TestRunConfigPrepare_EnableT2UnlimitedGood(t *testing.T) {
c := testConfig()
// Must have a T2 instance type if T2 Unlimited is enabled
c.InstanceType = "t2.micro"
c.EnableT2Unlimited = true
err := c.Prepare(nil)
if len(err) > 0 {
t.Fatalf("err: %s", err)
}
}
func TestRunConfigPrepare_EnableT2UnlimitedBadInstanceType(t *testing.T) {
c := testConfig()
// T2 Unlimited cannot be used with instance types other than T2
c.InstanceType = "m5.large"
c.EnableT2Unlimited = true
err := c.Prepare(nil)
if len(err) != 1 {
t.Fatalf("Should error if T2 Unlimited is enabled with non-T2 instance_type")
}
}
func TestRunConfigPrepare_EnableT2UnlimitedBadWithSpotInstanceRequest(t *testing.T) {
c := testConfig()
// T2 Unlimited cannot be used with Spot Instances
c.InstanceType = "t2.micro"
c.EnableT2Unlimited = true
c.SpotPrice = "auto"
c.SpotPriceAutoProduct = "Linux/UNIX"
err := c.Prepare(nil)
if len(err) != 1 {
t.Fatalf("Should error if T2 Unlimited has been used in conjuntion with a Spot Price request")
}
}
func TestRunConfigPrepare_SpotAuto(t *testing.T) {
c := testConfig()
c.SpotPrice = "auto"
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if spot_price_auto_product is not set and spot_price is set to auto")
}
// Good - SpotPrice and SpotPriceAutoProduct are correctly set
c.SpotPriceAutoProduct = "foo"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
c.SpotPrice = ""
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if spot_price is not set to auto and spot_price_auto_product is set")
c.SpotPriceAutoProduct = "foo"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
}
@@ -170,13 +119,12 @@ func TestRunConfigPrepare_UserData(t *testing.T) {
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserData = "foo"
c.UserDataFile = tf.Name()
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if user_data string and user_data_file have both been specified")
t.Fatalf("err: %s", err)
}
}
@@ -188,14 +136,13 @@ func TestRunConfigPrepare_UserDataFile(t *testing.T) {
c.UserDataFile = "idontexistidontthink"
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if the file specified by user_data_file does not exist")
t.Fatalf("err: %s", err)
}
tf, err := ioutil.TempFile("", "packer")
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserDataFile = tf.Name()
@@ -206,27 +153,27 @@ func TestRunConfigPrepare_UserDataFile(t *testing.T) {
func TestRunConfigPrepare_TemporaryKeyPairName(t *testing.T) {
c := testConfig()
c.Comm.SSHTemporaryKeyPairName = ""
c.TemporaryKeyPairName = ""
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.Comm.SSHTemporaryKeyPairName == "" {
if c.TemporaryKeyPairName == "" {
t.Fatal("keypair name is empty")
}
// Match prefix and UUID, e.g. "packer_5790d491-a0b8-c84c-c9d2-2aea55086550".
r := regexp.MustCompile(`\Apacker_(?:(?i)[a-f\d]{8}(?:-[a-f\d]{4}){3}-[a-f\d]{12}?)\z`)
if !r.MatchString(c.Comm.SSHTemporaryKeyPairName) {
if !r.MatchString(c.TemporaryKeyPairName) {
t.Fatal("keypair name is not valid")
}
c.Comm.SSHTemporaryKeyPairName = "ssh-key-123"
c.TemporaryKeyPairName = "ssh-key-123"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
if c.Comm.SSHTemporaryKeyPairName != "ssh-key-123" {
if c.TemporaryKeyPairName != "ssh-key-123" {
t.Fatal("keypair name does not match")
}
}
+58
View File
@@ -3,10 +3,15 @@ package common
import (
"errors"
"fmt"
"net"
"os"
"time"
"github.com/aws/aws-sdk-go/service/ec2"
packerssh "github.com/hashicorp/packer/communicator/ssh"
"github.com/hashicorp/packer/helper/multistep"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/agent"
)
type ec2Describer interface {
@@ -80,3 +85,56 @@ func SSHHost(e ec2Describer, sshInterface string) func(multistep.StateBag) (stri
return "", errors.New("couldn't determine address for instance")
}
}
// SSHConfig returns a function that can be used for the SSH communicator
// config for connecting to the instance created over SSH using the private key
// or password.
func SSHConfig(useAgent bool, username, password string) func(multistep.StateBag) (*ssh.ClientConfig, error) {
return func(state multistep.StateBag) (*ssh.ClientConfig, error) {
if useAgent {
authSock := os.Getenv("SSH_AUTH_SOCK")
if authSock == "" {
return nil, fmt.Errorf("SSH_AUTH_SOCK is not set")
}
sshAgent, err := net.Dial("unix", authSock)
if err != nil {
return nil, fmt.Errorf("Cannot connect to SSH Agent socket %q: %s", authSock, err)
}
return &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{
ssh.PublicKeysCallback(agent.NewClient(sshAgent).Signers),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}, nil
}
privateKey, hasKey := state.GetOk("privateKey")
if hasKey {
signer, err := ssh.ParsePrivateKey([]byte(privateKey.(string)))
if err != nil {
return nil, fmt.Errorf("Error setting up SSH config: %s", err)
}
return &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{
ssh.PublicKeys(signer),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}, nil
} else {
return &ssh.ClientConfig{
User: username,
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Auth: []ssh.AuthMethod{
ssh.Password(password),
ssh.KeyboardInteractive(
packerssh.PasswordKeyboardInteractive(password)),
}}, nil
}
}
}
+196 -299
View File
@@ -1,14 +1,16 @@
package common
import (
"errors"
"fmt"
"log"
"net"
"os"
"strconv"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
)
@@ -33,333 +35,228 @@ type StateChangeConf struct {
Target string
}
// Following are wrapper functions that use Packer's environment-variables to
// determing retry logic, then call the AWS SDK's built-in waiters.
func WaitUntilAMIAvailable(ctx aws.Context, conn *ec2.EC2, imageId string) error {
imageInput := ec2.DescribeImagesInput{
ImageIds: []*string{&imageId},
}
waitOpts := getWaiterOptions()
if len(waitOpts) == 0 {
// Bump this default to 30 minutes because the aws default
// of ten minutes doesn't work for some of our long-running copies.
waitOpts = append(waitOpts, request.WithWaiterMaxAttempts(120))
}
err := conn.WaitUntilImageAvailableWithContext(
ctx,
&imageInput,
waitOpts...)
return err
}
func WaitUntilInstanceTerminated(ctx aws.Context, conn *ec2.EC2, instanceId string) error {
instanceInput := ec2.DescribeInstancesInput{
InstanceIds: []*string{&instanceId},
}
err := conn.WaitUntilInstanceTerminatedWithContext(
ctx,
&instanceInput,
getWaiterOptions()...)
return err
}
// This function works for both requesting and cancelling spot instances.
func WaitUntilSpotRequestFulfilled(ctx aws.Context, conn *ec2.EC2, spotRequestId string) error {
spotRequestInput := ec2.DescribeSpotInstanceRequestsInput{
SpotInstanceRequestIds: []*string{&spotRequestId},
}
err := conn.WaitUntilSpotInstanceRequestFulfilledWithContext(
ctx,
&spotRequestInput,
getWaiterOptions()...)
return err
}
func WaitUntilVolumeAvailable(ctx aws.Context, conn *ec2.EC2, volumeId string) error {
volumeInput := ec2.DescribeVolumesInput{
VolumeIds: []*string{&volumeId},
}
err := conn.WaitUntilVolumeAvailableWithContext(
ctx,
&volumeInput,
getWaiterOptions()...)
return err
}
func WaitUntilSnapshotDone(ctx aws.Context, conn *ec2.EC2, snapshotID string) error {
snapInput := ec2.DescribeSnapshotsInput{
SnapshotIds: []*string{&snapshotID},
}
err := conn.WaitUntilSnapshotCompletedWithContext(
ctx,
&snapInput,
getWaiterOptions()...)
return err
}
// Wrappers for our custom AWS waiters
func WaitUntilVolumeAttached(ctx aws.Context, conn *ec2.EC2, volumeId string) error {
volumeInput := ec2.DescribeVolumesInput{
VolumeIds: []*string{&volumeId},
}
err := WaitForVolumeToBeAttached(conn,
ctx,
&volumeInput,
getWaiterOptions()...)
return err
}
func WaitUntilVolumeDetached(ctx aws.Context, conn *ec2.EC2, volumeId string) error {
volumeInput := ec2.DescribeVolumesInput{
VolumeIds: []*string{&volumeId},
}
err := WaitForVolumeToBeDetached(conn,
ctx,
&volumeInput,
getWaiterOptions()...)
return err
}
func WaitUntilImageImported(ctx aws.Context, conn *ec2.EC2, taskID string) error {
importInput := ec2.DescribeImportImageTasksInput{
ImportTaskIds: []*string{&taskID},
}
err := WaitForImageToBeImported(conn,
ctx,
&importInput,
getWaiterOptions()...)
return err
}
// Custom waiters using AWS's request.Waiter
func WaitForVolumeToBeAttached(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeVolumesInput, opts ...request.WaiterOption) error {
w := request.Waiter{
Name: "DescribeVolumes",
MaxAttempts: 40,
Delay: request.ConstantWaiterDelay(5 * time.Second),
Acceptors: []request.WaiterAcceptor{
{
State: request.SuccessWaiterState,
Matcher: request.PathAllWaiterMatch,
Argument: "Volumes[].Attachments[].State",
Expected: "attached",
},
},
Logger: c.Config.Logger,
NewRequest: func(opts []request.Option) (*request.Request, error) {
var inCpy *ec2.DescribeVolumesInput
if input != nil {
tmp := *input
inCpy = &tmp
// AMIStateRefreshFunc returns a StateRefreshFunc that is used to watch
// an AMI for state changes.
func AMIStateRefreshFunc(conn *ec2.EC2, imageId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeImages(&ec2.DescribeImagesInput{
ImageIds: []*string{&imageId},
})
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && ec2err.Code() == "InvalidAMIID.NotFound" {
// Set this to nil as if we didn't find anything.
resp = nil
} else if isTransientNetworkError(err) {
// Transient network error, treat it as if we didn't find anything
resp = nil
} else {
log.Printf("Error on AMIStateRefresh: %s", err)
return nil, "", err
}
req, _ := c.DescribeVolumesRequest(inCpy)
req.SetContext(ctx)
req.ApplyOptions(opts...)
return req, nil
},
}
w.ApplyOptions(opts...)
}
return w.WaitWithContext(ctx)
if resp == nil || len(resp.Images) == 0 {
// Sometimes AWS has consistency issues and doesn't see the
// AMI. Return an empty state.
return nil, "", nil
}
i := resp.Images[0]
return i, *i.State, nil
}
}
func WaitForVolumeToBeDetached(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeVolumesInput, opts ...request.WaiterOption) error {
w := request.Waiter{
Name: "DescribeVolumes",
MaxAttempts: 40,
Delay: request.ConstantWaiterDelay(5 * time.Second),
Acceptors: []request.WaiterAcceptor{
{
State: request.SuccessWaiterState,
Matcher: request.PathAllWaiterMatch,
Argument: "length(Volumes[].Attachments[]) == `0`",
Expected: true,
},
},
Logger: c.Config.Logger,
NewRequest: func(opts []request.Option) (*request.Request, error) {
var inCpy *ec2.DescribeVolumesInput
if input != nil {
tmp := *input
inCpy = &tmp
// InstanceStateRefreshFunc returns a StateRefreshFunc that is used to watch
// an EC2 instance.
func InstanceStateRefreshFunc(conn *ec2.EC2, instanceId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeInstances(&ec2.DescribeInstancesInput{
InstanceIds: []*string{&instanceId},
})
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && ec2err.Code() == "InvalidInstanceID.NotFound" {
// Set this to nil as if we didn't find anything.
resp = nil
} else if isTransientNetworkError(err) {
// Transient network error, treat it as if we didn't find anything
resp = nil
} else {
log.Printf("Error on InstanceStateRefresh: %s", err)
return nil, "", err
}
req, _ := c.DescribeVolumesRequest(inCpy)
req.SetContext(ctx)
req.ApplyOptions(opts...)
return req, nil
},
}
w.ApplyOptions(opts...)
}
return w.WaitWithContext(ctx)
if resp == nil || len(resp.Reservations) == 0 || len(resp.Reservations[0].Instances) == 0 {
// Sometimes AWS just has consistency issues and doesn't see
// our instance yet. Return an empty state.
return nil, "", nil
}
i := resp.Reservations[0].Instances[0]
return i, *i.State.Name, nil
}
}
func WaitForImageToBeImported(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeImportImageTasksInput, opts ...request.WaiterOption) error {
w := request.Waiter{
Name: "DescribeImages",
MaxAttempts: 720,
Delay: request.ConstantWaiterDelay(5 * time.Second),
Acceptors: []request.WaiterAcceptor{
{
State: request.SuccessWaiterState,
Matcher: request.PathAllWaiterMatch,
Argument: "ImportImageTasks[].Status",
Expected: "completed",
},
{
State: request.FailureWaiterState,
Matcher: request.PathAnyWaiterMatch,
Argument: "ImportImageTasks[].Status",
Expected: "deleted",
},
},
Logger: c.Config.Logger,
NewRequest: func(opts []request.Option) (*request.Request, error) {
var inCpy *ec2.DescribeImportImageTasksInput
if input != nil {
tmp := *input
inCpy = &tmp
// SpotRequestStateRefreshFunc returns a StateRefreshFunc that is used to watch
// a spot request for state changes.
func SpotRequestStateRefreshFunc(conn *ec2.EC2, spotRequestId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeSpotInstanceRequests(&ec2.DescribeSpotInstanceRequestsInput{
SpotInstanceRequestIds: []*string{&spotRequestId},
})
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && ec2err.Code() == "InvalidSpotInstanceRequestID.NotFound" {
// Set this to nil as if we didn't find anything.
resp = nil
} else if isTransientNetworkError(err) {
// Transient network error, treat it as if we didn't find anything
resp = nil
} else {
log.Printf("Error on SpotRequestStateRefresh: %s", err)
return nil, "", err
}
req, _ := c.DescribeImportImageTasksRequest(inCpy)
req.SetContext(ctx)
req.ApplyOptions(opts...)
return req, nil
},
}
if resp == nil || len(resp.SpotInstanceRequests) == 0 {
// Sometimes AWS has consistency issues and doesn't see the
// SpotRequest. Return an empty state.
return nil, "", nil
}
i := resp.SpotInstanceRequests[0]
return i, *i.State, nil
}
w.ApplyOptions(opts...)
return w.WaitWithContext(ctx)
}
// This helper function uses the environment variables AWS_TIMEOUT_SECONDS and
// AWS_POLL_DELAY_SECONDS to generate waiter options that can be passed into any
// request.Waiter function. These options will control how many times the waiter
// will retry the request, as well as how long to wait between the retries.
func ImportImageRefreshFunc(conn *ec2.EC2, importTaskId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeImportImageTasks(&ec2.DescribeImportImageTasksInput{
ImportTaskIds: []*string{
&importTaskId,
},
},
)
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && strings.HasPrefix(ec2err.Code(), "InvalidConversionTaskId") {
resp = nil
} else if isTransientNetworkError(err) {
resp = nil
} else {
log.Printf("Error on ImportImageRefresh: %s", err)
return nil, "", err
}
}
// DEFAULTING BEHAVIOR:
// if AWS_POLL_DELAY_SECONDS is set but the others are not, Packer will set this
// poll delay and use the waiter-specific default
if resp == nil || len(resp.ImportImageTasks) == 0 {
return nil, "", nil
}
// if AWS_TIMEOUT_SECONDS is set but AWS_MAX_ATTEMPTS is not, Packer will use
// AWS_TIMEOUT_SECONDS and _either_ AWS_POLL_DELAY_SECONDS _or_ 2 if the user has not set AWS_POLL_DELAY_SECONDS, to determine a max number of attempts to make.
// if AWS_TIMEOUT_SECONDS, _and_ AWS_MAX_ATTEMPTS are both set,
// AWS_TIMEOUT_SECONDS will be ignored.
// if AWS_MAX_ATTEMPTS is set but AWS_POLL_DELAY_SECONDS is not, then we will
// use waiter-specific defaults.
type envInfo struct {
envKey string
Val int
overridden bool
i := resp.ImportImageTasks[0]
return i, *i.Status, nil
}
}
type overridableWaitVars struct {
awsPollDelaySeconds envInfo
awsMaxAttempts envInfo
awsTimeoutSeconds envInfo
// WaitForState watches an object and waits for it to achieve a certain
// state.
func WaitForState(conf *StateChangeConf) (i interface{}, err error) {
log.Printf("Waiting for state to become: %s", conf.Target)
sleepSeconds := SleepSeconds()
maxTicks := TimeoutSeconds()/sleepSeconds + 1
notfoundTick := 0
for {
var currentState string
i, currentState, err = conf.Refresh()
if err != nil {
return
}
if i == nil {
// If we didn't find the resource, check if we have been
// not finding it for awhile, and if so, report an error.
notfoundTick += 1
if notfoundTick > maxTicks {
return nil, errors.New("couldn't find resource")
}
} else {
// Reset the counter for when a resource isn't found
notfoundTick = 0
if currentState == conf.Target {
return
}
if conf.StepState != nil {
if _, ok := conf.StepState.GetOk(multistep.StateCancelled); ok {
return nil, errors.New("interrupted")
}
}
found := false
for _, allowed := range conf.Pending {
if currentState == allowed {
found = true
break
}
}
if !found {
err := fmt.Errorf("unexpected state '%s', wanted target '%s'", currentState, conf.Target)
return nil, err
}
}
time.Sleep(time.Duration(sleepSeconds) * time.Second)
}
}
func getWaiterOptions() []request.WaiterOption {
envOverrides := getEnvOverrides()
waitOpts := applyEnvOverrides(envOverrides)
return waitOpts
func isTransientNetworkError(err error) bool {
if nerr, ok := err.(net.Error); ok && nerr.Temporary() {
return true
}
return false
}
func getOverride(varInfo envInfo) envInfo {
override := os.Getenv(varInfo.envKey)
// Returns 300 seconds (5 minutes) by default
// Some AWS operations, like copying an AMI to a distant region, take a very long time
// Allow user to override with AWS_TIMEOUT_SECONDS environment variable
func TimeoutSeconds() (seconds int) {
seconds = 300
override := os.Getenv("AWS_TIMEOUT_SECONDS")
if override != "" {
n, err := strconv.Atoi(override)
if err != nil {
log.Printf("Invalid %s '%s', using default", varInfo.envKey, override)
log.Printf("Invalid timeout seconds '%s', using default", override)
} else {
varInfo.overridden = true
varInfo.Val = n
seconds = n
}
}
return varInfo
}
func getEnvOverrides() overridableWaitVars {
// Load env vars from environment.
envValues := overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 0, false},
envInfo{"AWS_TIMEOUT_SECONDS", 0, false},
}
envValues.awsMaxAttempts = getOverride(envValues.awsMaxAttempts)
envValues.awsPollDelaySeconds = getOverride(envValues.awsPollDelaySeconds)
envValues.awsTimeoutSeconds = getOverride(envValues.awsTimeoutSeconds)
return envValues
log.Printf("Allowing %ds to complete (change with AWS_TIMEOUT_SECONDS)", seconds)
return seconds
}
func LogEnvOverrideWarnings() {
pollDelay := os.Getenv("AWS_POLL_DELAY_SECONDS")
timeoutSeconds := os.Getenv("AWS_TIMEOUT_SECONDS")
maxAttempts := os.Getenv("AWS_MAX_ATTEMPTS")
// Returns 2 seconds by default
// AWS async operations sometimes takes long times, if there are multiple parallel builds,
// polling at 2 second frequency will exceed the request limit. Allow 2 seconds to be
// overwritten with AWS_POLL_DELAY_SECONDS
func SleepSeconds() (seconds int) {
seconds = 2
if maxAttempts != "" && timeoutSeconds != "" {
warning := fmt.Sprintf("[WARNING] (aws): AWS_MAX_ATTEMPTS and " +
"AWS_TIMEOUT_SECONDS are both set. Packer will use " +
"AWS_MAX_ATTEMPTS and discard AWS_TIMEOUT_SECONDS.")
if pollDelay == "" {
warning = fmt.Sprintf("%s Since you have not set the poll delay, "+
"Packer will default to a 2-second delay.", warning)
override := os.Getenv("AWS_POLL_DELAY_SECONDS")
if override != "" {
n, err := strconv.Atoi(override)
if err != nil {
log.Printf("Invalid sleep seconds '%s', using default", override)
} else {
seconds = n
}
log.Printf(warning)
} else if timeoutSeconds != "" {
log.Printf("[WARNING] (aws): env var AWS_TIMEOUT_SECONDS is " +
"deprecated in favor of AWS_MAX_ATTEMPTS. If you have not " +
"explicitly set AWS_POLL_DELAY_SECONDS, we are defaulting to a " +
"poll delay of 2 seconds, regardless of the AWS waiter's default.")
}
if maxAttempts == "" && timeoutSeconds == "" && pollDelay == "" {
log.Printf("[INFO] (aws): No AWS timeout and polling overrides have been set. " +
"Packer will default to waiter-specific delays and timeouts. If you would " +
"like to customize the length of time between retries and max " +
"number of retries you may do so by setting the environment " +
"variables AWS_POLL_DELAY_SECONDS and AWS_MAX_ATTEMPTS to your " +
"desired values.")
}
}
func applyEnvOverrides(envOverrides overridableWaitVars) []request.WaiterOption {
waitOpts := make([]request.WaiterOption, 0)
// If user has set poll delay seconds, overwrite it. If user has NOT,
// default to a poll delay of 2 seconds
if envOverrides.awsPollDelaySeconds.overridden {
delaySeconds := request.ConstantWaiterDelay(time.Duration(envOverrides.awsPollDelaySeconds.Val) * time.Second)
waitOpts = append(waitOpts, request.WithWaiterDelay(delaySeconds))
}
// If user has set max attempts, overwrite it. If user hasn't set max
// attempts, default to whatever the waiter has set as a default.
if envOverrides.awsMaxAttempts.overridden {
waitOpts = append(waitOpts, request.WithWaiterMaxAttempts(envOverrides.awsMaxAttempts.Val))
} else if envOverrides.awsTimeoutSeconds.overridden {
maxAttempts := envOverrides.awsTimeoutSeconds.Val / envOverrides.awsPollDelaySeconds.Val
// override the delay so we can get the timeout right
if !envOverrides.awsPollDelaySeconds.overridden {
delaySeconds := request.ConstantWaiterDelay(time.Duration(envOverrides.awsPollDelaySeconds.Val) * time.Second)
waitOpts = append(waitOpts, request.WithWaiterDelay(delaySeconds))
}
waitOpts = append(waitOpts, request.WithWaiterMaxAttempts(maxAttempts))
}
return waitOpts
log.Printf("Using %ds as polling delay (change with AWS_POLL_DELAY_SECONDS)", seconds)
return seconds
}
-66
View File
@@ -1,66 +0,0 @@
package common
import (
"reflect"
"testing"
"time"
"github.com/aws/aws-sdk-go/aws/request"
)
func testGetWaiterOptions(t *testing.T) {
// no vars are set
envValues := overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 0, false},
envInfo{"AWS_TIMEOUT_SECONDS", 300, false},
}
options := applyEnvOverrides(envValues)
if len(options) > 0 {
t.Fatalf("Did not expect any waiter options to be generated; actual: %#v", options)
}
// all vars are set
envValues = overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 1, true},
envInfo{"AWS_MAX_ATTEMPTS", 800, true},
envInfo{"AWS_TIMEOUT_SECONDS", 20, true},
}
options = applyEnvOverrides(envValues)
expected := []request.WaiterOption{
request.WithWaiterDelay(request.ConstantWaiterDelay(time.Duration(1) * time.Second)),
request.WithWaiterMaxAttempts(800),
}
if !reflect.DeepEqual(options, expected) {
t.Fatalf("expected != actual!! Expected: %#v; Actual: %#v.", expected, options)
}
// poll delay is not set
envValues = overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 800, true},
envInfo{"AWS_TIMEOUT_SECONDS", 300, false},
}
options = applyEnvOverrides(envValues)
expected = []request.WaiterOption{
request.WithWaiterMaxAttempts(800),
}
if !reflect.DeepEqual(options, expected) {
t.Fatalf("expected != actual!! Expected: %#v; Actual: %#v.", expected, options)
}
// poll delay is not set but timeout seconds is
envValues = overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 0, false},
envInfo{"AWS_TIMEOUT_SECONDS", 20, true},
}
options = applyEnvOverrides(envValues)
expected = []request.WaiterOption{
request.WithWaiterDelay(request.ConstantWaiterDelay(time.Duration(2) * time.Second)),
request.WithWaiterMaxAttempts(10),
}
if !reflect.DeepEqual(options, expected) {
t.Fatalf("expected != actual!! Expected: %#v; Actual: %#v.", expected, options)
}
}
+14 -9
View File
@@ -7,7 +7,7 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
@@ -20,7 +20,7 @@ type StepAMIRegionCopy struct {
Name string
}
func (s *StepAMIRegionCopy) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepAMIRegionCopy) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
amis := state.Get("amis").(map[string]string)
@@ -53,7 +53,7 @@ func (s *StepAMIRegionCopy) Run(ctx context.Context, state multistep.StateBag) m
go func(region string) {
defer wg.Done()
id, snapshotIds, err := amiRegionCopy(ctx, state, s.AccessConfig, s.Name, ami, region, *ec2conn.Config.Region, regKeyID)
id, snapshotIds, err := amiRegionCopy(state, s.AccessConfig, s.Name, ami, region, *ec2conn.Config.Region, regKeyID)
lock.Lock()
defer lock.Unlock()
amis[region] = id
@@ -85,7 +85,7 @@ func (s *StepAMIRegionCopy) Cleanup(state multistep.StateBag) {
// amiRegionCopy does a copy for the given AMI to the target region and
// returns the resulting ID and snapshot IDs, or error.
func amiRegionCopy(ctx context.Context, state multistep.StateBag, config *AccessConfig, name string, imageId string,
func amiRegionCopy(state multistep.StateBag, config *AccessConfig, name string, imageId string,
target string, source string, keyID string) (string, []string, error) {
snapshotIds := []string{}
isEncrypted := false
@@ -100,9 +100,8 @@ func amiRegionCopy(ctx context.Context, state multistep.StateBag, config *Access
isEncrypted = true
}
regionconn := ec2.New(session.Copy(&aws.Config{
Region: aws.String(target),
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
}))
Region: aws.String(target)},
))
resp, err := regionconn.CopyImage(&ec2.CopyImageInput{
SourceRegion: &source,
@@ -117,8 +116,14 @@ func amiRegionCopy(ctx context.Context, state multistep.StateBag, config *Access
imageId, target, err)
}
// Wait for the image to become ready
if err := WaitUntilAMIAvailable(ctx, regionconn, *resp.ImageId); err != nil {
stateChange := StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: AMIStateRefreshFunc(regionconn, *resp.ImageId),
StepState: state,
}
if _, err := WaitForState(&stateChange); err != nil {
return "", snapshotIds, fmt.Errorf("Error waiting for AMI (%s) in region (%s): %s",
*resp.ImageId, target, err)
}
+1 -3
View File
@@ -9,7 +9,6 @@ import (
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
retry "github.com/hashicorp/packer/common"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
@@ -36,8 +35,7 @@ func (s *StepCreateTags) Run(_ context.Context, state multistep.StateBag) multis
ui.Say(fmt.Sprintf("Adding tags to AMI (%s)...", ami))
regionConn := ec2.New(session, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
Region: aws.String(region),
Region: aws.String(region),
})
// Retrieve image list for given AMI
+2 -4
View File
@@ -6,7 +6,6 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
@@ -38,9 +37,8 @@ func (s *StepDeregisterAMI) Run(_ context.Context, state multistep.StateBag) mul
}
regionconn := ec2.New(session.Copy(&aws.Config{
Region: aws.String(region),
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
}))
Region: aws.String(region)},
))
resp, err := regionconn.DescribeImages(&ec2.DescribeImagesInput{
Owners: aws.StringSlice([]string{"self"}),
+9 -2
View File
@@ -19,7 +19,7 @@ type StepCreateEncryptedAMICopy struct {
AMIMappings []BlockDevice
}
func (s *StepCreateEncryptedAMICopy) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepCreateEncryptedAMICopy) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
kmsKeyId := s.KeyID
@@ -65,8 +65,15 @@ func (s *StepCreateEncryptedAMICopy) Run(ctx context.Context, state multistep.St
}
// Wait for the copy to become ready
stateChange := StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: AMIStateRefreshFunc(ec2conn, *copyResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI copy to become ready...")
if err := WaitUntilAMIAvailable(ctx, ec2conn, *copyResp.ImageId); err != nil {
if _, err := WaitForState(&stateChange); err != nil {
err := fmt.Errorf("Error waiting for AMI Copy: %s", err)
state.Put("error", err)
ui.Error(err.Error())
+6 -9
View File
@@ -21,10 +21,9 @@ import (
// StepGetPassword reads the password from a Windows server and sets it
// on the WinRM config.
type StepGetPassword struct {
Debug bool
Comm *communicator.Config
Timeout time.Duration
BuildName string
Debug bool
Comm *communicator.Config
Timeout time.Duration
}
func (s *StepGetPassword) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
@@ -95,21 +94,19 @@ WaitLoop:
"Password (since debug is enabled): %s", s.Comm.WinRMPassword))
}
// store so that we can access this later during provisioning
commonhelper.SetSharedState("winrm_password", s.Comm.WinRMPassword, s.BuildName)
packer.LogSecretFilter.Set(s.Comm.WinRMPassword)
commonhelper.SetSharedState("winrm_password", s.Comm.WinRMPassword)
return multistep.ActionContinue
}
func (s *StepGetPassword) Cleanup(multistep.StateBag) {
commonhelper.RemoveSharedStateFile("winrm_password", s.BuildName)
commonhelper.RemoveSharedStateFile("winrm_password")
}
func (s *StepGetPassword) waitForPassword(state multistep.StateBag, cancel <-chan struct{}) (string, error) {
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
privateKey := s.Comm.SSHPrivateKey
privateKey := state.Get("privateKey").(string)
for {
select {
+26 -20
View File
@@ -3,19 +3,22 @@ package common
import (
"context"
"fmt"
"io/ioutil"
"os"
"runtime"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
type StepKeyPair struct {
Debug bool
Comm *communicator.Config
DebugKeyPath string
Debug bool
SSHAgentAuth bool
DebugKeyPath string
TemporaryKeyPairName string
KeyPairName string
PrivateKeyFile string
doCleanup bool
}
@@ -23,40 +26,43 @@ type StepKeyPair struct {
func (s *StepKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if s.Comm.SSHPrivateKeyFile != "" {
if s.PrivateKeyFile != "" {
ui.Say("Using existing SSH private key")
privateKeyBytes, err := s.Comm.ReadSSHPrivateKeyFile()
privateKeyBytes, err := ioutil.ReadFile(s.PrivateKeyFile)
if err != nil {
state.Put("error", err)
state.Put("error", fmt.Errorf(
"Error loading configured private key file: %s", err))
return multistep.ActionHalt
}
s.Comm.SSHPrivateKey = privateKeyBytes
state.Put("keyPair", s.KeyPairName)
state.Put("privateKey", string(privateKeyBytes))
return multistep.ActionContinue
}
if s.Comm.SSHAgentAuth && s.Comm.SSHKeyPairName == "" {
if s.SSHAgentAuth && s.KeyPairName == "" {
ui.Say("Using SSH Agent with key pair in Source AMI")
return multistep.ActionContinue
}
if s.Comm.SSHAgentAuth && s.Comm.SSHKeyPairName != "" {
ui.Say(fmt.Sprintf("Using SSH Agent for existing key pair %s", s.Comm.SSHKeyPairName))
if s.SSHAgentAuth && s.KeyPairName != "" {
ui.Say(fmt.Sprintf("Using SSH Agent for existing key pair %s", s.KeyPairName))
state.Put("keyPair", s.KeyPairName)
return multistep.ActionContinue
}
if s.Comm.SSHTemporaryKeyPairName == "" {
if s.TemporaryKeyPairName == "" {
ui.Say("Not using temporary keypair")
s.Comm.SSHKeyPairName = ""
state.Put("keyPair", "")
return multistep.ActionContinue
}
ec2conn := state.Get("ec2").(*ec2.EC2)
ui.Say(fmt.Sprintf("Creating temporary keypair: %s", s.Comm.SSHTemporaryKeyPairName))
ui.Say(fmt.Sprintf("Creating temporary keypair: %s", s.TemporaryKeyPairName))
keyResp, err := ec2conn.CreateKeyPair(&ec2.CreateKeyPairInput{
KeyName: &s.Comm.SSHTemporaryKeyPairName})
KeyName: &s.TemporaryKeyPairName})
if err != nil {
state.Put("error", fmt.Errorf("Error creating temporary keypair: %s", err))
return multistep.ActionHalt
@@ -64,9 +70,9 @@ func (s *StepKeyPair) Run(_ context.Context, state multistep.StateBag) multistep
s.doCleanup = true
// Set some data for use in future steps
s.Comm.SSHKeyPairName = s.Comm.SSHTemporaryKeyPairName
s.Comm.SSHPrivateKey = []byte(*keyResp.KeyMaterial)
// Set some state data for use in future steps
state.Put("keyPair", s.TemporaryKeyPairName)
state.Put("privateKey", *keyResp.KeyMaterial)
// If we're in debug mode, output the private key to the working
// directory.
@@ -107,10 +113,10 @@ func (s *StepKeyPair) Cleanup(state multistep.StateBag) {
// Remove the keypair
ui.Say("Deleting temporary keypair...")
_, err := ec2conn.DeleteKeyPair(&ec2.DeleteKeyPairInput{KeyName: &s.Comm.SSHTemporaryKeyPairName})
_, err := ec2conn.DeleteKeyPair(&ec2.DeleteKeyPairInput{KeyName: &s.TemporaryKeyPairName})
if err != nil {
ui.Error(fmt.Sprintf(
"Error cleaning up keypair. Please delete the key manually: %s", s.Comm.SSHTemporaryKeyPairName))
"Error cleaning up keypair. Please delete the key manually: %s", s.TemporaryKeyPairName))
}
// Also remove the physical key if we're debugging.
@@ -7,7 +7,6 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
@@ -146,8 +145,7 @@ func (s *StepModifyAMIAttributes) Run(_ context.Context, state multistep.StateBa
for region, ami := range amis {
ui.Say(fmt.Sprintf("Modifying attributes on AMI (%s)...", ami))
regionConn := ec2.New(session, &aws.Config{
Region: aws.String(region),
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
Region: aws.String(region),
})
for name, input := range options {
ui.Message(fmt.Sprintf("Modifying: %s", name))
@@ -167,8 +165,7 @@ func (s *StepModifyAMIAttributes) Run(_ context.Context, state multistep.StateBa
for _, snapshot := range region_snapshots {
ui.Say(fmt.Sprintf("Modifying attributes on snapshot (%s)...", snapshot))
regionConn := ec2.New(session, &aws.Config{
Region: aws.String(region),
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
Region: aws.String(region),
})
for name, input := range snapshotOptions {
ui.Message(fmt.Sprintf("Modifying: %s", name))
@@ -3,7 +3,6 @@ package common
import (
"context"
"fmt"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
@@ -12,7 +11,7 @@ import (
)
type StepModifyEBSBackedInstance struct {
EnableAMIENASupport *bool
EnableAMIENASupport bool
EnableAMISriovNetSupport bool
}
@@ -38,22 +37,16 @@ func (s *StepModifyEBSBackedInstance) Run(_ context.Context, state multistep.Sta
}
}
// Handle EnaSupport flag.
// Set EnaSupport to true.
// As of February 2017, this applies to C5, I3, P2, R4, X1, and m4.16xlarge
if s.EnableAMIENASupport != nil {
var prefix string
if *s.EnableAMIENASupport {
prefix = "En"
} else {
prefix = "Dis"
}
ui.Say(fmt.Sprintf("%sabling Enhanced Networking (ENA)...", prefix))
if s.EnableAMIENASupport {
ui.Say("Enabling Enhanced Networking (ENA)...")
_, err := ec2conn.ModifyInstanceAttribute(&ec2.ModifyInstanceAttributeInput{
InstanceId: instance.InstanceId,
EnaSupport: &ec2.AttributeBooleanValue{Value: aws.Bool(*s.EnableAMIENASupport)},
EnaSupport: &ec2.AttributeBooleanValue{Value: aws.Bool(true)},
})
if err != nil {
err := fmt.Errorf("Error %sabling Enhanced Networking (ENA) on %s: %s", strings.ToLower(prefix), *instance.InstanceId, err)
err := fmt.Errorf("Error enabling Enhanced Networking (ENA) on %s: %s", *instance.InstanceId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
-154
View File
@@ -1,154 +0,0 @@
package common
import (
"context"
"fmt"
"log"
"math/rand"
"sort"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepNetworkInfo queries AWS for information about
// VPC's and Subnets that is used throughout the AMI creation process.
//
// Produces (adding them to the state bag):
// vpc_id string - the VPC ID
// subnet_id string - the Subnet ID
// availability_zone string - the AZ name
type StepNetworkInfo struct {
VpcId string
VpcFilter VpcFilterOptions
SubnetId string
SubnetFilter SubnetFilterOptions
AvailabilityZone string
SecurityGroupIds []string
SecurityGroupFilter SecurityGroupFilterOptions
}
type subnetsSort []*ec2.Subnet
func (a subnetsSort) Len() int { return len(a) }
func (a subnetsSort) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a subnetsSort) Less(i, j int) bool {
return *a[i].AvailableIpAddressCount < *a[j].AvailableIpAddressCount
}
// Returns the most recent AMI out of a slice of images.
func mostFreeSubnet(subnets []*ec2.Subnet) *ec2.Subnet {
sortedSubnets := subnets
sort.Sort(subnetsSort(sortedSubnets))
return sortedSubnets[len(sortedSubnets)-1]
}
func (s *StepNetworkInfo) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
// VPC
if s.VpcId == "" && !s.VpcFilter.Empty() {
params := &ec2.DescribeVpcsInput{}
params.Filters = buildEc2Filters(s.VpcFilter.Filters)
s.VpcFilter.Filters[aws.String("state")] = aws.String("available")
log.Printf("Using VPC Filters %v", params)
vpcResp, err := ec2conn.DescribeVpcs(params)
if err != nil {
err := fmt.Errorf("Error querying VPCs: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(vpcResp.Vpcs) != 1 {
err := fmt.Errorf("Exactly one VPC should match the filter, but %d VPC's was found matching filters: %v", len(vpcResp.Vpcs), params)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
s.VpcId = *vpcResp.Vpcs[0].VpcId
ui.Message(fmt.Sprintf("Found VPC ID: %s", s.VpcId))
}
// Subnet
if s.SubnetId == "" && !s.SubnetFilter.Empty() {
params := &ec2.DescribeSubnetsInput{}
s.SubnetFilter.Filters[aws.String("state")] = aws.String("available")
if s.VpcId != "" {
s.SubnetFilter.Filters[aws.String("vpc-id")] = &s.VpcId
}
if s.AvailabilityZone != "" {
s.SubnetFilter.Filters[aws.String("availability-zone")] = &s.AvailabilityZone
}
params.Filters = buildEc2Filters(s.SubnetFilter.Filters)
log.Printf("Using Subnet Filters %v", params)
subnetsResp, err := ec2conn.DescribeSubnets(params)
if err != nil {
err := fmt.Errorf("Error querying Subnets: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(subnetsResp.Subnets) == 0 {
err := fmt.Errorf("No Subnets was found matching filters: %v", params)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(subnetsResp.Subnets) > 1 && !s.SubnetFilter.Random && !s.SubnetFilter.MostFree {
err := fmt.Errorf("Your filter matched %d Subnets. Please try a more specific search, or set random or most_free to true.", len(subnetsResp.Subnets))
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
var subnet *ec2.Subnet
switch {
case s.SubnetFilter.MostFree:
subnet = mostFreeSubnet(subnetsResp.Subnets)
case s.SubnetFilter.Random:
subnet = subnetsResp.Subnets[rand.Intn(len(subnetsResp.Subnets))]
default:
subnet = subnetsResp.Subnets[0]
}
s.SubnetId = *subnet.SubnetId
ui.Message(fmt.Sprintf("Found Subnet ID: %s", s.SubnetId))
}
// Try to find AZ and VPC Id from Subnet if they are not yet found/given
if s.SubnetId != "" && (s.AvailabilityZone == "" || s.VpcId == "") {
log.Printf("[INFO] Finding AZ and VpcId for the given subnet '%s'", s.SubnetId)
resp, err := ec2conn.DescribeSubnets(&ec2.DescribeSubnetsInput{SubnetIds: []*string{&s.SubnetId}})
if err != nil {
err := fmt.Errorf("Describing the subnet: %s returned error: %s.", s.SubnetId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if s.AvailabilityZone == "" {
s.AvailabilityZone = *resp.Subnets[0].AvailabilityZone
log.Printf("[INFO] AvailabilityZone found: '%s'", s.AvailabilityZone)
}
if s.VpcId == "" {
s.VpcId = *resp.Subnets[0].VpcId
log.Printf("[INFO] VpcId found: '%s'", s.VpcId)
}
}
state.Put("vpc_id", s.VpcId)
state.Put("availability_zone", s.AvailabilityZone)
state.Put("subnet_id", s.SubnetId)
return multistep.ActionContinue
}
func (s *StepNetworkInfo) Cleanup(multistep.StateBag) {}
@@ -12,7 +12,6 @@ import (
"github.com/aws/aws-sdk-go/service/ec2"
retry "github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
@@ -20,18 +19,18 @@ import (
type StepRunSourceInstance struct {
AssociatePublicIpAddress bool
AvailabilityZone string
BlockDevices BlockDevices
Comm *communicator.Config
Ctx interpolate.Context
Debug bool
EbsOptimized bool
EnableT2Unlimited bool
ExpectedRootDevice string
IamInstanceProfile string
InstanceInitiatedShutdownBehavior string
InstanceType string
IsRestricted bool
SourceAMI string
SubnetId string
Tags TagMap
UserData string
UserDataFile string
@@ -40,9 +39,12 @@ type StepRunSourceInstance struct {
instanceId string
}
func (s *StepRunSourceInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepRunSourceInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
var keyName string
if name, ok := state.GetOk("keyPair"); ok {
keyName = name.(string)
}
securityGroupIds := aws.StringSlice(state.Get("securityGroupIds").([]string))
ui := state.Get("ui").(packer.Ui)
@@ -102,7 +104,6 @@ func (s *StepRunSourceInstance) Run(ctx context.Context, state multistep.StateBa
return multistep.ActionHalt
}
az := state.Get("availability_zone").(string)
runOpts := &ec2.RunInstancesInput{
ImageId: &s.SourceAMI,
InstanceType: &s.InstanceType,
@@ -111,15 +112,10 @@ func (s *StepRunSourceInstance) Run(ctx context.Context, state multistep.StateBa
MinCount: aws.Int64(1),
IamInstanceProfile: &ec2.IamInstanceProfileSpecification{Name: &s.IamInstanceProfile},
BlockDeviceMappings: s.BlockDevices.BuildLaunchDevices(),
Placement: &ec2.Placement{AvailabilityZone: &az},
Placement: &ec2.Placement{AvailabilityZone: &s.AvailabilityZone},
EbsOptimized: &s.EbsOptimized,
}
if s.EnableT2Unlimited {
creditOption := "unlimited"
runOpts.CreditSpecification = &ec2.CreditSpecificationRequest{CpuCredits: &creditOption}
}
// Collect tags for tagging on resource creation
var tagSpecs []*ec2.TagSpecification
@@ -148,24 +144,22 @@ func (s *StepRunSourceInstance) Run(ctx context.Context, state multistep.StateBa
volTags.Report(ui)
}
if s.Comm.SSHKeyPairName != "" {
runOpts.KeyName = &s.Comm.SSHKeyPairName
if keyName != "" {
runOpts.KeyName = &keyName
}
subnetId := state.Get("subnet_id").(string)
if subnetId != "" && s.AssociatePublicIpAddress {
if s.SubnetId != "" && s.AssociatePublicIpAddress {
runOpts.NetworkInterfaces = []*ec2.InstanceNetworkInterfaceSpecification{
{
DeviceIndex: aws.Int64(0),
AssociatePublicIpAddress: &s.AssociatePublicIpAddress,
SubnetId: aws.String(subnetId),
SubnetId: &s.SubnetId,
Groups: securityGroupIds,
DeleteOnTermination: aws.Bool(true),
},
}
} else {
runOpts.SubnetId = aws.String(subnetId)
runOpts.SubnetId = &s.SubnetId
runOpts.SecurityGroupIds = securityGroupIds
}
@@ -191,7 +185,7 @@ func (s *StepRunSourceInstance) Run(ctx context.Context, state multistep.StateBa
describeInstance := &ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
}
if err := ec2conn.WaitUntilInstanceRunningWithContext(ctx, describeInstance); err != nil {
if err := ec2conn.WaitUntilInstanceRunning(describeInstance); err != nil {
err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", instanceId, err)
state.Put("error", err)
ui.Error(err.Error())
@@ -303,8 +297,14 @@ func (s *StepRunSourceInstance) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error terminating instance, may still be around: %s", err))
return
}
stateChange := StateChangeConf{
Pending: []string{"pending", "running", "shutting-down", "stopped", "stopping"},
Refresh: InstanceStateRefreshFunc(ec2conn, s.instanceId),
Target: "terminated",
}
if err := WaitUntilInstanceTerminated(aws.BackgroundContext(), ec2conn, s.instanceId); err != nil {
_, err := WaitForState(&stateChange)
if err != nil {
ui.Error(err.Error())
}
}
+42 -61
View File
@@ -14,7 +14,6 @@ import (
"github.com/aws/aws-sdk-go/service/ec2"
retry "github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
@@ -22,10 +21,9 @@ import (
type StepRunSpotInstance struct {
AssociatePublicIpAddress bool
AvailabilityZone string
BlockDevices BlockDevices
BlockDurationMinutes int64
Debug bool
Comm *communicator.Config
EbsOptimized bool
ExpectedRootDevice string
IamInstanceProfile string
@@ -34,7 +32,7 @@ type StepRunSpotInstance struct {
SourceAMI string
SpotPrice string
SpotPriceProduct string
SpotTags TagMap
SubnetId string
Tags TagMap
VolumeTags TagMap
UserData string
@@ -45,8 +43,12 @@ type StepRunSpotInstance struct {
spotRequest *ec2.SpotInstanceRequest
}
func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepRunSpotInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
var keyName string
if name, ok := state.GetOk("keyPair"); ok {
keyName = name.(string)
}
securityGroupIds := aws.StringSlice(state.Get("securityGroupIds").([]string))
ui := state.Get("ui").(packer.Ui)
@@ -84,12 +86,7 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}
spotPrice := s.SpotPrice
azConfig := ""
if azRaw, ok := state.GetOk("availability_zone"); ok {
azConfig = azRaw.(string)
}
az := azConfig
availabilityZone := s.AvailabilityZone
if spotPrice == "auto" {
ui.Message(fmt.Sprintf(
"Finding spot price for %s %s...",
@@ -100,7 +97,7 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
resp, err := ec2conn.DescribeSpotPriceHistory(&ec2.DescribeSpotPriceHistoryInput{
InstanceTypes: []*string{&s.InstanceType},
ProductDescriptions: []*string{&s.SpotPriceProduct},
AvailabilityZone: &az,
AvailabilityZone: &s.AvailabilityZone,
StartTime: &startTime,
})
if err != nil {
@@ -120,8 +117,8 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}
if price == 0 || current < price {
price = current
if azConfig == "" {
az = *history.AvailabilityZone
if s.AvailabilityZone == "" {
availabilityZone = *history.AvailabilityZone
}
}
}
@@ -165,41 +162,35 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
UserData: &userData,
IamInstanceProfile: &ec2.IamInstanceProfileSpecification{Name: &s.IamInstanceProfile},
Placement: &ec2.SpotPlacement{
AvailabilityZone: &az,
AvailabilityZone: &availabilityZone,
},
BlockDeviceMappings: s.BlockDevices.BuildLaunchDevices(),
EbsOptimized: &s.EbsOptimized,
}
subnetId := state.Get("subnet_id").(string)
if subnetId != "" && s.AssociatePublicIpAddress {
if s.SubnetId != "" && s.AssociatePublicIpAddress {
runOpts.NetworkInterfaces = []*ec2.InstanceNetworkInterfaceSpecification{
{
DeviceIndex: aws.Int64(0),
AssociatePublicIpAddress: &s.AssociatePublicIpAddress,
SubnetId: &subnetId,
SubnetId: &s.SubnetId,
Groups: securityGroupIds,
DeleteOnTermination: aws.Bool(true),
},
}
} else {
runOpts.SubnetId = &subnetId
runOpts.SubnetId = &s.SubnetId
runOpts.SecurityGroupIds = securityGroupIds
}
if s.Comm.SSHKeyPairName != "" {
runOpts.KeyName = &s.Comm.SSHKeyPairName
}
spotInstanceInput := &ec2.RequestSpotInstancesInput{
LaunchSpecification: runOpts,
SpotPrice: &spotPrice,
}
if s.BlockDurationMinutes != 0 {
spotInstanceInput.BlockDurationMinutes = &s.BlockDurationMinutes
if keyName != "" {
runOpts.KeyName = &keyName
}
runSpotResp, err := ec2conn.RequestSpotInstances(spotInstanceInput)
runSpotResp, err := ec2conn.RequestSpotInstances(&ec2.RequestSpotInstancesInput{
SpotPrice: &spotPrice,
LaunchSpecification: runOpts,
})
if err != nil {
err := fmt.Errorf("Error launching source spot instance: %s", err)
state.Put("error", err)
@@ -211,7 +202,13 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
spotRequestId := s.spotRequest.SpotInstanceRequestId
ui.Message(fmt.Sprintf("Waiting for spot request (%s) to become active...", *spotRequestId))
err = WaitUntilSpotRequestFulfilled(ctx, ec2conn, *spotRequestId)
stateChange := StateChangeConf{
Pending: []string{"open"},
Target: "active",
Refresh: SpotRequestStateRefreshFunc(ec2conn, *spotRequestId),
StepState: state,
}
_, err = WaitForState(&stateChange)
if err != nil {
err := fmt.Errorf("Error waiting for spot request (%s) to become ready: %s", *spotRequestId, err)
state.Put("error", err)
@@ -230,33 +227,6 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}
instanceId = *spotResp.SpotInstanceRequests[0].InstanceId
// Tag spot instance request
spotTags, err := s.SpotTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging spot request: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
spotTags.Report(ui)
if len(spotTags) > 0 && s.SpotTags.IsSet() {
// Retry creating tags for about 2.5 minutes
err = retry.Retry(0.2, 30, 11, func(_ uint) (bool, error) {
_, err := ec2conn.CreateTags(&ec2.CreateTagsInput{
Tags: spotTags,
Resources: []*string{spotRequestId},
})
return true, err
})
if err != nil {
err := fmt.Errorf("Error tagging spot request: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
}
// Set the instance ID so that the cleanup works properly
s.instanceId = instanceId
@@ -265,7 +235,7 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
describeInstance := &ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
}
if err := ec2conn.WaitUntilInstanceRunningWithContext(ctx, describeInstance); err != nil {
if err := ec2conn.WaitUntilInstanceRunning(describeInstance); err != nil {
err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", instanceId, err)
state.Put("error", err)
ui.Error(err.Error())
@@ -374,8 +344,13 @@ func (s *StepRunSpotInstance) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error cancelling the spot request, may still be around: %s", err))
return
}
stateChange := StateChangeConf{
Pending: []string{"active", "open"},
Refresh: SpotRequestStateRefreshFunc(ec2conn, *s.spotRequest.SpotInstanceRequestId),
Target: "cancelled",
}
err := WaitUntilSpotRequestFulfilled(aws.BackgroundContext(), ec2conn, *s.spotRequest.SpotInstanceRequestId)
_, err := WaitForState(&stateChange)
if err != nil {
ui.Error(err.Error())
}
@@ -389,8 +364,14 @@ func (s *StepRunSpotInstance) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error terminating instance, may still be around: %s", err))
return
}
stateChange := StateChangeConf{
Pending: []string{"pending", "running", "shutting-down", "stopped", "stopping"},
Refresh: InstanceStateRefreshFunc(ec2conn, s.instanceId),
Target: "terminated",
}
if err := WaitUntilInstanceTerminated(aws.BackgroundContext(), ec2conn, s.instanceId); err != nil {
_, err := WaitForState(&stateChange)
if err != nil {
ui.Error(err.Error())
}
}
+36 -62
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"log"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
@@ -18,8 +17,8 @@ import (
type StepSecurityGroup struct {
CommConfig *communicator.Config
SecurityGroupFilter SecurityGroupFilterOptions
SecurityGroupIds []string
VpcId string
TemporarySGSourceCidr string
createdGroupId string
@@ -28,7 +27,6 @@ type StepSecurityGroup struct {
func (s *StepSecurityGroup) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
vpcId := state.Get("vpc_id").(string)
if len(s.SecurityGroupIds) > 0 {
_, err := ec2conn.DescribeSecurityGroups(
@@ -47,35 +45,6 @@ func (s *StepSecurityGroup) Run(_ context.Context, state multistep.StateBag) mul
return multistep.ActionContinue
}
if !s.SecurityGroupFilter.Empty() {
params := &ec2.DescribeSecurityGroupsInput{}
if vpcId != "" {
s.SecurityGroupFilter.Filters[aws.String("vpc-id")] = &vpcId
}
params.Filters = buildEc2Filters(s.SecurityGroupFilter.Filters)
log.Printf("Using SecurityGroup Filters %v", params)
sgResp, err := ec2conn.DescribeSecurityGroups(params)
if err != nil {
err := fmt.Errorf("Couldn't find security groups for filter: %s", err)
log.Printf("[DEBUG] %s", err.Error())
state.Put("error", err)
return multistep.ActionHalt
}
securityGroupIds := []string{}
for _, sg := range sgResp.SecurityGroups {
securityGroupIds = append(securityGroupIds, *sg.GroupId)
}
ui.Message(fmt.Sprintf("Found Security Group(s): %s", strings.Join(securityGroupIds, ", ")))
state.Put("securityGroupIds", securityGroupIds)
return multistep.ActionContinue
}
port := s.CommConfig.Port()
if port == 0 {
if s.CommConfig.Type != "none" {
@@ -91,7 +60,9 @@ func (s *StepSecurityGroup) Run(_ context.Context, state multistep.StateBag) mul
Description: aws.String("Temporary group for Packer"),
}
group.VpcId = &vpcId
if s.VpcId != "" {
group.VpcId = &s.VpcId
}
groupResp, err := ec2conn.CreateSecurityGroup(group)
if err != nil {
@@ -103,7 +74,38 @@ func (s *StepSecurityGroup) Run(_ context.Context, state multistep.StateBag) mul
// Set the group ID so we can delete it later
s.createdGroupId = *groupResp.GroupId
// Wait for the security group become available for authorizing
// Authorize the SSH access for the security group
req := &ec2.AuthorizeSecurityGroupIngressInput{
GroupId: groupResp.GroupId,
IpProtocol: aws.String("tcp"),
FromPort: aws.Int64(int64(port)),
ToPort: aws.Int64(int64(port)),
CidrIp: aws.String(s.TemporarySGSourceCidr),
}
// We loop and retry this a few times because sometimes the security
// group isn't available immediately because AWS resources are eventually
// consistent.
ui.Say(fmt.Sprintf(
"Authorizing access to port %d from %s in the temporary security group...",
port, s.TemporarySGSourceCidr))
for i := 0; i < 5; i++ {
_, err = ec2conn.AuthorizeSecurityGroupIngress(req)
if err == nil {
break
}
log.Printf("Error authorizing. Will sleep and retry. %s", err)
time.Sleep((time.Duration(i) * time.Second) + 1)
}
if err != nil {
err := fmt.Errorf("Error creating temporary security group: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
log.Printf("[DEBUG] Waiting for temporary security group: %s", s.createdGroupId)
err = waitUntilSecurityGroupExists(ec2conn,
&ec2.DescribeSecurityGroupsInput{
@@ -119,34 +121,6 @@ func (s *StepSecurityGroup) Run(_ context.Context, state multistep.StateBag) mul
return multistep.ActionHalt
}
// Authorize the SSH access for the security group
groupRules := &ec2.AuthorizeSecurityGroupIngressInput{
GroupId: groupResp.GroupId,
IpPermissions: []*ec2.IpPermission{
{
FromPort: aws.Int64(int64(port)),
ToPort: aws.Int64(int64(port)),
IpRanges: []*ec2.IpRange{
{
CidrIp: aws.String(s.TemporarySGSourceCidr),
},
},
IpProtocol: aws.String("tcp"),
},
},
}
ui.Say(fmt.Sprintf(
"Authorizing access to port %d from %s in the temporary security group...",
port, s.TemporarySGSourceCidr))
_, err = ec2conn.AuthorizeSecurityGroupIngress(groupRules)
if err != nil {
err := fmt.Errorf("Error authorizing temporary security group: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
// Set some state data for use in future steps
state.Put("securityGroupIds", []string{s.createdGroupId})
+19 -23
View File
@@ -20,11 +20,22 @@ import (
type StepSourceAMIInfo struct {
SourceAmi string
EnableAMISriovNetSupport bool
EnableAMIENASupport *bool
AMIVirtType string
EnableAMIENASupport bool
AmiFilters AmiFilterOptions
}
// Build a slice of AMI filter options from the filters provided.
func buildAmiFilters(input map[*string]*string) []*ec2.Filter {
var filters []*ec2.Filter
for k, v := range input {
filters = append(filters, &ec2.Filter{
Name: k,
Values: []*string{v},
})
}
return filters
}
type imageSort []*ec2.Image
func (a imageSort) Len() int { return len(a) }
@@ -54,7 +65,7 @@ func (s *StepSourceAMIInfo) Run(_ context.Context, state multistep.StateBag) mul
// We have filters to apply
if len(s.AmiFilters.Filters) > 0 {
params.Filters = buildEc2Filters(s.AmiFilters.Filters)
params.Filters = buildAmiFilters(s.AmiFilters.Filters)
}
if len(s.AmiFilters.Owners) > 0 {
params.Owners = s.AmiFilters.Owners
@@ -94,13 +105,11 @@ func (s *StepSourceAMIInfo) Run(_ context.Context, state multistep.StateBag) mul
// Enhanced Networking can only be enabled on HVM AMIs.
// See http://goo.gl/icuXh5
if (s.EnableAMIENASupport != nil && *s.EnableAMIENASupport) || s.EnableAMISriovNetSupport {
err = s.canEnableEnhancedNetworking(image)
if err != nil {
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if (s.EnableAMIENASupport || s.EnableAMISriovNetSupport) && *image.VirtualizationType != "hvm" {
err := fmt.Errorf("Cannot enable enhanced networking, source AMI '%s' is not HVM", s.SourceAmi)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
state.Put("source_image", image)
@@ -108,16 +117,3 @@ func (s *StepSourceAMIInfo) Run(_ context.Context, state multistep.StateBag) mul
}
func (s *StepSourceAMIInfo) Cleanup(multistep.StateBag) {}
func (s *StepSourceAMIInfo) canEnableEnhancedNetworking(image *ec2.Image) error {
if s.AMIVirtType == "hvm" {
return nil
}
if s.AMIVirtType != "" {
return fmt.Errorf("Cannot enable enhanced networking, AMIVirtType '%s' is not HVM", s.AMIVirtType)
}
if *image.VirtualizationType != "hvm" {
return fmt.Errorf("Cannot enable enhanced networking, source AMI '%s' is not HVM", s.SourceAmi)
}
return nil
}
@@ -1,42 +0,0 @@
package common
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/stretchr/testify/assert"
"testing"
)
func TestStepSourceAmiInfo_PVImage(t *testing.T) {
err := new(StepSourceAMIInfo).canEnableEnhancedNetworking(&ec2.Image{
VirtualizationType: aws.String("paravirtual"),
})
assert.Error(t, err)
}
func TestStepSourceAmiInfo_HVMImage(t *testing.T) {
err := new(StepSourceAMIInfo).canEnableEnhancedNetworking(&ec2.Image{
VirtualizationType: aws.String("hvm"),
})
assert.NoError(t, err)
}
func TestStepSourceAmiInfo_PVImageWithAMIVirtPV(t *testing.T) {
stepSourceAMIInfo := StepSourceAMIInfo{
AMIVirtType: "paravirtual",
}
err := stepSourceAMIInfo.canEnableEnhancedNetworking(&ec2.Image{
VirtualizationType: aws.String("paravirtual"),
})
assert.Error(t, err)
}
func TestStepSourceAmiInfo_PVImageWithAMIVirtHVM(t *testing.T) {
stepSourceAMIInfo := StepSourceAMIInfo{
AMIVirtType: "hvm",
}
err := stepSourceAMIInfo.canEnableEnhancedNetworking(&ec2.Image{
VirtualizationType: aws.String("paravirtual"),
})
assert.NoError(t, err)
}
@@ -16,7 +16,7 @@ type StepStopEBSBackedInstance struct {
DisableStopInstance bool
}
func (s *StepStopEBSBackedInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepStopEBSBackedInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
ui := state.Get("ui").(packer.Ui)
@@ -78,11 +78,9 @@ func (s *StepStopEBSBackedInstance) Run(ctx context.Context, state multistep.Sta
// Wait for the instance to actually stop
ui.Say("Waiting for the instance to stop...")
err = ec2conn.WaitUntilInstanceStoppedWithContext(ctx,
&ec2.DescribeInstancesInput{
InstanceIds: []*string{instance.InstanceId},
},
getWaiterOptions()...)
err = ec2conn.WaitUntilInstanceStopped(&ec2.DescribeInstancesInput{
InstanceIds: []*string{instance.InstanceId},
})
if err != nil {
err := fmt.Errorf("Error waiting for instance to stop: %s", err)
+42 -44
View File
@@ -9,11 +9,9 @@ import (
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
@@ -50,7 +48,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"ami_description",
"run_tags",
"run_volume_tags",
"spot_tags",
"snapshot_tags",
"tags",
},
@@ -72,7 +69,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs = packer.MultiErrorAppend(errs, b.config.BlockDevices.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
if b.config.IsSpotInstance() && ((b.config.AMIENASupport != nil && *b.config.AMIENASupport) || b.config.AMISriovNetSupport) {
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
@@ -83,29 +80,38 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packer.Artifact, error) {
session, err := b.config.Session()
if err != nil {
return nil, err
}
if !b.config.AMISkipRegionValidation {
regionsToValidate := append(b.config.AMIRegions, b.config.RawRegion)
err := b.config.AccessConfig.ValidateRegion(regionsToValidate...)
ec2conn := ec2.New(session)
// If the subnet is specified but not the VpcId or AZ, try to determine them automatically
if b.config.SubnetId != "" && (b.config.AvailabilityZone == "" || b.config.VpcId == "") {
log.Printf("[INFO] Finding AZ and VpcId for the given subnet '%s'", b.config.SubnetId)
resp, err := ec2conn.DescribeSubnets(&ec2.DescribeSubnetsInput{SubnetIds: []*string{&b.config.SubnetId}})
if err != nil {
return nil, fmt.Errorf("error validating regions: %v", err)
return nil, err
}
if b.config.AvailabilityZone == "" {
b.config.AvailabilityZone = *resp.Subnets[0].AvailabilityZone
log.Printf("[INFO] AvailabilityZone found: '%s'", b.config.AvailabilityZone)
}
if b.config.VpcId == "" {
b.config.VpcId = *resp.Subnets[0].VpcId
log.Printf("[INFO] VpcId found: '%s'", b.config.VpcId)
}
}
ec2conn := ec2.New(session, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
})
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("config", b.config)
state.Put("ec2", ec2conn)
state.Put("awsSession", session)
state.Put("hook", hook)
@@ -116,10 +122,9 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
BlockDurationMinutes: b.config.BlockDurationMinutes,
Ctx: b.config.ctx,
Comm: &b.config.RunConfig.Comm,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
ExpectedRootDevice: "ebs",
@@ -129,7 +134,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -138,18 +143,18 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
} else {
instanceStep = &awscommon.StepRunSourceInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -168,49 +173,41 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
EnableAMIENASupport: b.config.AMIENASupport,
AmiFilters: b.config.SourceAmiFilter,
AMIVirtType: b.config.AMIVirtType,
},
&awscommon.StepNetworkInfo{
VpcId: b.config.VpcId,
VpcFilter: b.config.VpcFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
AvailabilityZone: b.config.AvailabilityZone,
},
&awscommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
Debug: b.config.PackerDebug,
SSHAgentAuth: b.config.Comm.SSHAgentAuth,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
KeyPairName: b.config.SSHKeyPairName,
TemporaryKeyPairName: b.config.TemporaryKeyPairName,
PrivateKeyFile: b.config.RunConfig.Comm.SSHPrivateKey,
},
&awscommon.StepSecurityGroup{
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
VpcId: b.config.VpcId,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
&awscommon.StepCleanupVolumes{
&stepCleanupVolumes{
BlockDevices: b.config.BlockDevices,
},
instanceStep,
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
b.config.RunConfig.Comm.SSHPassword),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&awscommon.StepStopEBSBackedInstance{
Skip: b.config.IsSpotInstance(),
DisableStopInstance: b.config.DisableStopInstance,
@@ -259,6 +256,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
// Run!
b.runner = common.NewRunner(steps, b.config.PackerConfig, ui)
b.runner.Run(state)
// If there was an error, return that
if rawErr, ok := state.GetOk("error"); ok {
return nil, rawErr.(error)
-2
View File
@@ -47,7 +47,6 @@ func TestBuilderPrepare_AMIName(t *testing.T) {
// Test good
config["ami_name"] = "foo"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
@@ -100,7 +99,6 @@ func TestBuilderPrepare_InvalidShutdownBehavior(t *testing.T) {
// Test good
config["shutdown_behavior"] = "terminate"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
@@ -1,4 +1,4 @@
package common
package ebs
import (
"context"
@@ -6,6 +6,7 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
@@ -13,16 +14,16 @@ import (
// stepCleanupVolumes cleans up any orphaned volumes that were not designated to
// remain after termination of the instance. These volumes are typically ones
// that are marked as "delete on terminate:false" in the source_ami of a build.
type StepCleanupVolumes struct {
BlockDevices BlockDevices
type stepCleanupVolumes struct {
BlockDevices common.BlockDevices
}
func (s *StepCleanupVolumes) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
func (s *stepCleanupVolumes) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
// stepCleanupVolumes is for Cleanup only
return multistep.ActionContinue
}
func (s *StepCleanupVolumes) Cleanup(state multistep.StateBag) {
func (s *stepCleanupVolumes) Cleanup(state multistep.StateBag) {
ec2conn := state.Get("ec2").(*ec2.EC2)
instanceRaw := state.Get("instance")
var instance *ec2.Instance
+13 -24
View File
@@ -3,11 +3,9 @@ package ebs
import (
"context"
"fmt"
"log"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common/random"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
@@ -16,24 +14,17 @@ type stepCreateAMI struct {
image *ec2.Image
}
func (s *stepCreateAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
func (s *stepCreateAMI) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
ui := state.Get("ui").(packer.Ui)
// Create the image
amiName := config.AMIName
if config.AMIEncryptBootVolume {
// to avoid having a temporary unencrypted
// image named config.AMIName
amiName = random.AlphaNum(7)
}
ui.Say(fmt.Sprintf("Creating unencrypted AMI %s from instance %s", amiName, *instance.InstanceId))
ui.Say(fmt.Sprintf("Creating the AMI: %s", config.AMIName))
createOpts := &ec2.CreateImageInput{
InstanceId: instance.InstanceId,
Name: &amiName,
Name: &config.AMIName,
BlockDeviceMappings: config.BlockDevices.BuildAMIDevices(),
}
@@ -52,18 +43,16 @@ func (s *stepCreateAMI) Run(ctx context.Context, state multistep.StateBag) multi
state.Put("amis", amis)
// Wait for the image to become ready
ui.Say("Waiting for AMI to become ready...")
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *createResp.ImageId); err != nil {
log.Printf("Error waiting for AMI: %s", err)
imagesResp, err := ec2conn.DescribeImages(&ec2.DescribeImagesInput{ImageIds: []*string{createResp.ImageId}})
if err != nil {
log.Printf("Unable to determine reason waiting for AMI failed: %s", err)
err = fmt.Errorf("Unknown error waiting for AMI.")
} else {
stateReason := imagesResp.Images[0].StateReason
err = fmt.Errorf("Error waiting for AMI. Reason: %s", stateReason)
}
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *createResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if _, err := awscommon.WaitForState(&stateChange); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
+41 -41
View File
@@ -7,11 +7,9 @@ import (
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
@@ -50,7 +48,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"run_tags",
"run_volume_tags",
"snapshot_tags",
"spot_tags",
"tags",
},
},
@@ -87,7 +84,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("no volume with name '%s' is found", b.config.RootDevice.SourceDeviceName))
}
if b.config.IsSpotInstance() && ((b.config.AMIENASupport != nil && *b.config.AMIENASupport) || b.config.AMISriovNetSupport) {
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
@@ -98,7 +95,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
@@ -107,9 +104,24 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if err != nil {
return nil, err
}
ec2conn := ec2.New(session, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
})
ec2conn := ec2.New(session)
// If the subnet is specified but not the VpcId or AZ, try to determine them automatically
if b.config.SubnetId != "" && (b.config.AvailabilityZone == "" || b.config.VpcId == "") {
log.Printf("[INFO] Finding AZ and VpcId for the given subnet '%s'", b.config.SubnetId)
resp, err := ec2conn.DescribeSubnets(&ec2.DescribeSubnetsInput{SubnetIds: []*string{&b.config.SubnetId}})
if err != nil {
return nil, err
}
if b.config.AvailabilityZone == "" {
b.config.AvailabilityZone = *resp.Subnets[0].AvailabilityZone
log.Printf("[INFO] AvailabilityZone found: '%s'", b.config.AvailabilityZone)
}
if b.config.VpcId == "" {
b.config.VpcId = *resp.Subnets[0].VpcId
log.Printf("[INFO] VpcId found: '%s'", b.config.VpcId)
}
}
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
@@ -124,10 +136,9 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
BlockDurationMinutes: b.config.BlockDurationMinutes,
Ctx: b.config.ctx,
Comm: &b.config.RunConfig.Comm,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
ExpectedRootDevice: "ebs",
@@ -137,7 +148,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -146,18 +157,18 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
} else {
instanceStep = &awscommon.StepRunSourceInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -179,49 +190,38 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
EnableAMIENASupport: b.config.AMIENASupport,
AmiFilters: b.config.SourceAmiFilter,
AMIVirtType: b.config.AMIVirtType,
},
&awscommon.StepNetworkInfo{
VpcId: b.config.VpcId,
VpcFilter: b.config.VpcFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
AvailabilityZone: b.config.AvailabilityZone,
},
&awscommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
Debug: b.config.PackerDebug,
SSHAgentAuth: b.config.Comm.SSHAgentAuth,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
KeyPairName: b.config.SSHKeyPairName,
TemporaryKeyPairName: b.config.TemporaryKeyPairName,
PrivateKeyFile: b.config.RunConfig.Comm.SSHPrivateKey,
},
&awscommon.StepSecurityGroup{
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
VpcId: b.config.VpcId,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
&awscommon.StepCleanupVolumes{
BlockDevices: b.config.BlockDevices,
},
instanceStep,
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
b.config.RunConfig.Comm.SSHPassword),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&awscommon.StepStopEBSBackedInstance{
Skip: b.config.IsSpotInstance(),
DisableStopInstance: b.config.DisableStopInstance,
@@ -16,12 +16,12 @@ type StepRegisterAMI struct {
RootDevice RootBlockDevice
AMIDevices []*ec2.BlockDeviceMapping
LaunchDevices []*ec2.BlockDeviceMapping
EnableAMIENASupport *bool
EnableAMIENASupport bool
EnableAMISriovNetSupport bool
image *ec2.Image
}
func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepRegisterAMI) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
snapshotIds := state.Get("snapshot_ids").(map[string]string)
@@ -44,7 +44,7 @@ func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) mul
// As of February 2017, this applies to C3, C4, D2, I2, R3, and M4 (excluding m4.16xlarge)
registerOpts.SriovNetSupport = aws.String("simple")
}
if s.EnableAMIENASupport != nil && *s.EnableAMIENASupport {
if s.EnableAMIENASupport {
// Set EnaSupport to true
// As of February 2017, this applies to C5, I3, P2, R4, X1, and m4.16xlarge
registerOpts.EnaSupport = aws.Bool(true)
@@ -63,8 +63,15 @@ func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) mul
state.Put("amis", amis)
// Wait for the image to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *registerResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *registerResp.ImageId); err != nil {
if _, err := awscommon.WaitForState(&stateChange); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
@@ -2,6 +2,7 @@ package ebssurrogate
import (
"context"
"errors"
"fmt"
"sync"
"time"
@@ -22,7 +23,7 @@ type StepSnapshotVolumes struct {
snapshotIds map[string]string
}
func (s *StepSnapshotVolumes) snapshotVolume(ctx context.Context, deviceName string, state multistep.StateBag) error {
func (s *StepSnapshotVolumes) snapshotVolume(deviceName string, state multistep.StateBag) error {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ec2.Instance)
@@ -51,12 +52,33 @@ func (s *StepSnapshotVolumes) snapshotVolume(ctx context.Context, deviceName str
// Set the snapshot ID so we can delete it later
s.snapshotIds[deviceName] = *createSnapResp.SnapshotId
// Wait for snapshot to be created
err = awscommon.WaitUntilSnapshotDone(ctx, ec2conn, *createSnapResp.SnapshotId)
// Wait for the snapshot to be ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
StepState: state,
Target: "completed",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeSnapshots(&ec2.DescribeSnapshotsInput{
SnapshotIds: []*string{createSnapResp.SnapshotId},
})
if err != nil {
return nil, "", err
}
if len(resp.Snapshots) == 0 {
return nil, "", errors.New("No snapshots found.")
}
s := resp.Snapshots[0]
return s, *s.State, nil
},
}
_, err = awscommon.WaitForState(&stateChange)
return err
}
func (s *StepSnapshotVolumes) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepSnapshotVolumes) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
s.snapshotIds = map[string]string{}
@@ -67,7 +89,7 @@ func (s *StepSnapshotVolumes) Run(ctx context.Context, state multistep.StateBag)
wg.Add(1)
go func(device *ec2.BlockDeviceMapping) {
defer wg.Done()
if err := s.snapshotVolume(ctx, *device.DeviceName, state); err != nil {
if err := s.snapshotVolume(*device.DeviceName, state); err != nil {
errs = multierror.Append(errs, err)
}
}(device)
+49 -49
View File
@@ -6,11 +6,9 @@ import (
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
@@ -26,7 +24,7 @@ type Config struct {
awscommon.RunConfig `mapstructure:",squash"`
VolumeMappings []BlockDevice `mapstructure:"ebs_volumes"`
AMIENASupport *bool `mapstructure:"ena_support"`
AMIENASupport bool `mapstructure:"ena_support"`
AMISriovNetSupport bool `mapstructure:"sriov_support"`
launchBlockDevices awscommon.BlockDevices
@@ -38,22 +36,17 @@ type Builder struct {
runner multistep.Runner
}
type EngineVarsTemplate struct {
BuildRegion string
SourceAMI string
}
func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
b.config.ctx.Funcs = awscommon.TemplateFuncs
// Create passthrough for {{ .BuildRegion }} and {{ .SourceAMI }} variables
// so we can fill them in later
b.config.ctx.Data = &EngineVarsTemplate{
BuildRegion: `{{ .BuildRegion }}`,
SourceAMI: `{{ .SourceAMI }} `,
}
err := config.Decode(&b.config, &config.DecodeOpts{
Interpolate: true,
InterpolateContext: &b.config.ctx,
InterpolateFilter: &interpolate.RenderFilter{
Exclude: []string{
"run_tags",
"ebs_volumes",
},
},
}, raws...)
if err != nil {
return nil, err
@@ -76,7 +69,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs = packer.MultiErrorAppend(errs, err)
}
if b.config.IsSpotInstance() && ((b.config.AMIENASupport != nil && *b.config.AMIENASupport) || b.config.AMISriovNetSupport) {
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
@@ -87,7 +80,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
@@ -96,13 +89,28 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if err != nil {
return nil, err
}
ec2conn := ec2.New(session, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
})
ec2conn := ec2.New(session)
// If the subnet is specified but not the VpcId or AZ, try to determine them automatically
if b.config.SubnetId != "" && (b.config.AvailabilityZone == "" || b.config.VpcId == "") {
log.Printf("[INFO] Finding AZ and VpcId for the given subnet '%s'", b.config.SubnetId)
resp, err := ec2conn.DescribeSubnets(&ec2.DescribeSubnetsInput{SubnetIds: []*string{&b.config.SubnetId}})
if err != nil {
return nil, err
}
if b.config.AvailabilityZone == "" {
b.config.AvailabilityZone = *resp.Subnets[0].AvailabilityZone
log.Printf("[INFO] AvailabilityZone found: '%s'", b.config.AvailabilityZone)
}
if b.config.VpcId == "" {
b.config.VpcId = *resp.Subnets[0].VpcId
log.Printf("[INFO] VpcId found: '%s'", b.config.VpcId)
}
}
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("config", b.config)
state.Put("ec2", ec2conn)
state.Put("hook", hook)
state.Put("ui", ui)
@@ -112,10 +120,9 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.launchBlockDevices,
BlockDurationMinutes: b.config.BlockDurationMinutes,
Ctx: b.config.ctx,
Comm: &b.config.RunConfig.Comm,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
ExpectedRootDevice: "ebs",
@@ -125,7 +132,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -133,18 +140,18 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
} else {
instanceStep = &awscommon.StepRunSourceInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.launchBlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -159,24 +166,18 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
EnableAMIENASupport: b.config.AMIENASupport,
AmiFilters: b.config.SourceAmiFilter,
},
&awscommon.StepNetworkInfo{
VpcId: b.config.VpcId,
VpcFilter: b.config.VpcFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
AvailabilityZone: b.config.AvailabilityZone,
},
&awscommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
Debug: b.config.PackerDebug,
SSHAgentAuth: b.config.Comm.SSHAgentAuth,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
KeyPairName: b.config.SSHKeyPairName,
TemporaryKeyPairName: b.config.TemporaryKeyPairName,
PrivateKeyFile: b.config.RunConfig.Comm.SSHPrivateKey,
},
&awscommon.StepSecurityGroup{
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
VpcId: b.config.VpcId,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
instanceStep,
@@ -185,22 +186,21 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
Ctx: b.config.ctx,
},
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
b.config.RunConfig.Comm.SSHPassword),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&awscommon.StepStopEBSBackedInstance{
Skip: b.config.IsSpotInstance(),
DisableStopInstance: b.config.DisableStopInstance,
-1
View File
@@ -61,7 +61,6 @@ func TestBuilderPrepare_InvalidShutdownBehavior(t *testing.T) {
// Test good
config["shutdown_behavior"] = "terminate"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
+42 -38
View File
@@ -9,11 +9,9 @@ import (
"os"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
@@ -71,7 +69,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"run_volume_tags",
"snapshot_tags",
"tags",
"spot_tags",
},
},
}, configs...)
@@ -158,7 +155,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs, fmt.Errorf("x509_key_path points to bad file: %s", err))
}
if b.config.IsSpotInstance() && ((b.config.AMIENASupport != nil && *b.config.AMIENASupport) || b.config.AMISriovNetSupport) {
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
@@ -168,7 +165,8 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
@@ -177,9 +175,24 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if err != nil {
return nil, err
}
ec2conn := ec2.New(session, &aws.Config{
HTTPClient: commonhelper.HttpClientWithEnvironmentProxy(),
})
ec2conn := ec2.New(session)
// If the subnet is specified but not the VpcId or AZ, try to determine them automatically
if b.config.SubnetId != "" && (b.config.AvailabilityZone == "" || b.config.VpcId == "") {
log.Printf("[INFO] Finding AZ and VpcId for the given subnet '%s'", b.config.SubnetId)
resp, err := ec2conn.DescribeSubnets(&ec2.DescribeSubnetsInput{SubnetIds: []*string{&b.config.SubnetId}})
if err != nil {
return nil, err
}
if b.config.AvailabilityZone == "" {
b.config.AvailabilityZone = *resp.Subnets[0].AvailabilityZone
log.Printf("[INFO] AvailabilityZone found: '%s'", b.config.AvailabilityZone)
}
if b.config.VpcId == "" {
b.config.VpcId = *resp.Subnets[0].VpcId
log.Printf("[INFO] VpcId found: '%s'", b.config.VpcId)
}
}
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
@@ -194,10 +207,9 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
BlockDurationMinutes: b.config.BlockDurationMinutes,
Ctx: b.config.ctx,
Comm: &b.config.RunConfig.Comm,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
IamInstanceProfile: b.config.IamInstanceProfile,
@@ -205,24 +217,24 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
SpotTags: b.config.SpotTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
}
} else {
instanceStep = &awscommon.StepRunSourceInstance{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
@@ -240,46 +252,38 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
EnableAMIENASupport: b.config.AMIENASupport,
AmiFilters: b.config.SourceAmiFilter,
AMIVirtType: b.config.AMIVirtType,
},
&awscommon.StepNetworkInfo{
VpcId: b.config.VpcId,
VpcFilter: b.config.VpcFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
AvailabilityZone: b.config.AvailabilityZone,
},
&awscommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
Debug: b.config.PackerDebug,
SSHAgentAuth: b.config.Comm.SSHAgentAuth,
DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
KeyPairName: b.config.SSHKeyPairName,
PrivateKeyFile: b.config.RunConfig.Comm.SSHPrivateKey,
TemporaryKeyPairName: b.config.TemporaryKeyPairName,
},
&awscommon.StepSecurityGroup{
CommConfig: &b.config.RunConfig.Comm,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
SecurityGroupIds: b.config.SecurityGroupIds,
VpcId: b.config.VpcId,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
instanceStep,
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
b.config.RunConfig.Comm.SSHPassword),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&StepUploadX509Cert{},
&StepBundleVolume{
Debug: b.config.PackerDebug,
+11 -42
View File
@@ -8,13 +8,13 @@ import (
"github.com/hashicorp/packer/packer"
)
func testConfig() (config map[string]interface{}, tf *os.File) {
func testConfig() map[string]interface{} {
tf, err := ioutil.TempFile("", "packer")
if err != nil {
panic(err)
}
config = map[string]interface{}{
return map[string]interface{}{
"account_id": "foo",
"ami_name": "foo",
"instance_type": "m1.small",
@@ -26,8 +26,6 @@ func testConfig() (config map[string]interface{}, tf *os.File) {
"x509_key_path": tf.Name(),
"x509_upload_path": "/foo",
}
return config, tf
}
func TestBuilder_ImplementsBuilder(t *testing.T) {
@@ -40,11 +38,7 @@ func TestBuilder_ImplementsBuilder(t *testing.T) {
func TestBuilderPrepare_AccountId(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
config["account_id"] = ""
warnings, err := b.Prepare(config)
@@ -80,13 +74,10 @@ func TestBuilderPrepare_AccountId(t *testing.T) {
func TestBuilderPrepare_AMIName(t *testing.T) {
var b Builder
config, tempfile := testConfig()
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
// Test good
config["ami_name"] = "foo"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
@@ -120,10 +111,7 @@ func TestBuilderPrepare_AMIName(t *testing.T) {
func TestBuilderPrepare_BundleDestination(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
config["bundle_destination"] = ""
warnings, err := b.Prepare(config)
@@ -141,10 +129,7 @@ func TestBuilderPrepare_BundleDestination(t *testing.T) {
func TestBuilderPrepare_BundlePrefix(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
@@ -161,9 +146,7 @@ func TestBuilderPrepare_BundlePrefix(t *testing.T) {
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config, tempfile := testConfig()
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
@@ -178,10 +161,7 @@ func TestBuilderPrepare_InvalidKey(t *testing.T) {
func TestBuilderPrepare_S3Bucket(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
config["s3_bucket"] = ""
warnings, err := b.Prepare(config)
@@ -204,10 +184,7 @@ func TestBuilderPrepare_S3Bucket(t *testing.T) {
func TestBuilderPrepare_X509CertPath(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
config["x509_cert_path"] = ""
warnings, err := b.Prepare(config)
@@ -232,7 +209,6 @@ func TestBuilderPrepare_X509CertPath(t *testing.T) {
t.Fatalf("error tempfile: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
config["x509_cert_path"] = tf.Name()
warnings, err = b.Prepare(config)
@@ -246,10 +222,7 @@ func TestBuilderPrepare_X509CertPath(t *testing.T) {
func TestBuilderPrepare_X509KeyPath(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
config["x509_key_path"] = ""
warnings, err := b.Prepare(config)
@@ -274,7 +247,6 @@ func TestBuilderPrepare_X509KeyPath(t *testing.T) {
t.Fatalf("error tempfile: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
config["x509_key_path"] = tf.Name()
warnings, err = b.Prepare(config)
@@ -288,10 +260,7 @@ func TestBuilderPrepare_X509KeyPath(t *testing.T) {
func TestBuilderPrepare_X509UploadPath(t *testing.T) {
b := &Builder{}
config, tempfile := testConfig()
config["skip_region_validation"] = true
defer os.Remove(tempfile.Name())
defer tempfile.Close()
config := testConfig()
config["x509_upload_path"] = ""
warnings, err := b.Prepare(config)
+11 -4
View File
@@ -12,11 +12,11 @@ import (
)
type StepRegisterAMI struct {
EnableAMIENASupport *bool
EnableAMIENASupport bool
EnableAMISriovNetSupport bool
}
func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
func (s *StepRegisterAMI) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
manifestPath := state.Get("remote_manifest_path").(string)
@@ -38,7 +38,7 @@ func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) mul
// As of February 2017, this applies to C3, C4, D2, I2, R3, and M4 (excluding m4.16xlarge)
registerOpts.SriovNetSupport = aws.String("simple")
}
if s.EnableAMIENASupport != nil && *s.EnableAMIENASupport {
if s.EnableAMIENASupport {
// Set EnaSupport to true
// As of February 2017, this applies to C5, I3, P2, R4, X1, and m4.16xlarge
registerOpts.EnaSupport = aws.Bool(true)
@@ -58,8 +58,15 @@ func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) mul
state.Put("amis", amis)
// Wait for the image to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *registerResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *registerResp.ImageId); err != nil {
if _, err := awscommon.WaitForState(&stateChange); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
+1 -1
View File
@@ -4,7 +4,7 @@ Here's a list of things we like to get done in no particular order:
- [ ] Blob/image rename post-processor
- [ ] SSH to private ip through subnet
- [ ] chroot builder
- [ ] support cross-storage account image source (i.e. pre-build blob copy)
- [ ] support cross-storage account image source (ie. pre-build blob copy)
- [ ] look up object id when using device code (graph api /me ?)
- [ ] device flow support for Windows
- [x] look up tenant id in all cases (see device flow code)
@@ -90,7 +90,7 @@
"image": {
"uri": "https://localhost/custom.vhd"
},
"name": "[parameters('osDiskName')]",
"name": "osdisk",
"osType": "Linux",
"vhd": {
"uri": "[concat(parameters('storageAccountBlobEndpoint'),variables('vmStorageAccountContainerName'),'/', parameters('osDiskName'),'.vhd')]"
+9 -31
View File
@@ -18,9 +18,6 @@ type AdditionalDiskArtifact struct {
}
type Artifact struct {
// OS type: Linux, Windows
OSType string
// VHD
StorageAccountLocation string
OSDiskUri string
@@ -29,30 +26,23 @@ type Artifact struct {
TemplateUriReadOnlySas string
// Managed Image
ManagedImageResourceGroupName string
ManagedImageName string
ManagedImageLocation string
ManagedImageId string
ManagedImageOSDiskSnapshotName string
ManagedImageDataDiskSnapshotPrefix string
ManagedImageResourceGroupName string
ManagedImageName string
ManagedImageLocation string
// Additional Disks
AdditionalDisks *[]AdditionalDiskArtifact
}
func NewManagedImageArtifact(osType, resourceGroup, name, location, id, osDiskSnapshotName, dataDiskSnapshotPrefix string) (*Artifact, error) {
func NewManagedImageArtifact(resourceGroup, name, location string) (*Artifact, error) {
return &Artifact{
ManagedImageResourceGroupName: resourceGroup,
ManagedImageName: name,
ManagedImageLocation: location,
ManagedImageId: id,
OSType: osType,
ManagedImageOSDiskSnapshotName: osDiskSnapshotName,
ManagedImageDataDiskSnapshotPrefix: dataDiskSnapshotPrefix,
ManagedImageResourceGroupName: resourceGroup,
ManagedImageName: name,
ManagedImageLocation: location,
}, nil
}
func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string, osType string) (*Artifact, error) {
func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string) (*Artifact, error) {
if template == nil {
return nil, fmt.Errorf("nil capture template")
}
@@ -86,7 +76,6 @@ func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string,
}
return &Artifact{
OSType: osType,
OSDiskUri: vhdUri.String(),
OSDiskUriReadOnlySas: getSasUrl(getStorageUrlPath(vhdUri)),
TemplateUri: templateUri.String(),
@@ -137,10 +126,7 @@ func (*Artifact) Files() []string {
}
func (a *Artifact) Id() string {
if a.OSDiskUri != "" {
return a.OSDiskUri
}
return a.ManagedImageId
return a.OSDiskUri
}
func (a *Artifact) State(name string) interface{} {
@@ -156,18 +142,10 @@ func (a *Artifact) String() string {
var buf bytes.Buffer
buf.WriteString(fmt.Sprintf("%s:\n\n", a.BuilderId()))
buf.WriteString(fmt.Sprintf("OSType: %s\n", a.OSType))
if a.isManagedImage() {
buf.WriteString(fmt.Sprintf("ManagedImageResourceGroupName: %s\n", a.ManagedImageResourceGroupName))
buf.WriteString(fmt.Sprintf("ManagedImageName: %s\n", a.ManagedImageName))
buf.WriteString(fmt.Sprintf("ManagedImageId: %s\n", a.ManagedImageId))
buf.WriteString(fmt.Sprintf("ManagedImageLocation: %s\n", a.ManagedImageLocation))
if a.ManagedImageOSDiskSnapshotName != "" {
buf.WriteString(fmt.Sprintf("ManagedImageOSDiskSnapshotName: %s\n", a.ManagedImageOSDiskSnapshotName))
}
if a.ManagedImageDataDiskSnapshotPrefix != "" {
buf.WriteString(fmt.Sprintf("ManagedImageDataDiskSnapshotPrefix: %s\n", a.ManagedImageDataDiskSnapshotPrefix))
}
} else {
buf.WriteString(fmt.Sprintf("StorageAccountLocation: %s\n", a.StorageAccountLocation))
buf.WriteString(fmt.Sprintf("OSDiskUri: %s\n", a.OSDiskUri))
+9 -88
View File
@@ -10,7 +10,7 @@ func getFakeSasUrl(name string) string {
return fmt.Sprintf("SAS-%s", name)
}
func TestArtifactIdVHD(t *testing.T) {
func TestArtifactId(t *testing.T) {
template := CaptureTemplate{
Resources: []CaptureResources{
{
@@ -28,7 +28,7 @@ func TestArtifactIdVHD(t *testing.T) {
},
}
artifact, err := NewArtifact(&template, getFakeSasUrl, "Linux")
artifact, err := NewArtifact(&template, getFakeSasUrl)
if err != nil {
t.Fatalf("err=%s", err)
}
@@ -41,73 +41,6 @@ func TestArtifactIdVHD(t *testing.T) {
}
}
func TestArtifactIDManagedImage(t *testing.T) {
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "fakeDataDiskSnapshotPrefix")
if err != nil {
t.Fatalf("err=%s", err)
}
expected := `Azure.ResourceManagement.VMImage:
OSType: Linux
ManagedImageResourceGroupName: fakeResourceGroup
ManagedImageName: fakeName
ManagedImageId: fakeID
ManagedImageLocation: fakeLocation
ManagedImageOSDiskSnapshotName: fakeOsDiskSnapshotName
ManagedImageDataDiskSnapshotPrefix: fakeDataDiskSnapshotPrefix
`
result := artifact.String()
if result != expected {
t.Fatalf("bad: %s", result)
}
}
func TestArtifactIDManagedImageWithoutOSDiskSnapshotName(t *testing.T) {
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "", "fakeDataDiskSnapshotPrefix")
if err != nil {
t.Fatalf("err=%s", err)
}
expected := `Azure.ResourceManagement.VMImage:
OSType: Linux
ManagedImageResourceGroupName: fakeResourceGroup
ManagedImageName: fakeName
ManagedImageId: fakeID
ManagedImageLocation: fakeLocation
ManagedImageDataDiskSnapshotPrefix: fakeDataDiskSnapshotPrefix
`
result := artifact.String()
if result != expected {
t.Fatalf("bad: %s", result)
}
}
func TestArtifactIDManagedImageWithoutDataDiskSnapshotPrefix(t *testing.T) {
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "")
if err != nil {
t.Fatalf("err=%s", err)
}
expected := `Azure.ResourceManagement.VMImage:
OSType: Linux
ManagedImageResourceGroupName: fakeResourceGroup
ManagedImageName: fakeName
ManagedImageId: fakeID
ManagedImageLocation: fakeLocation
ManagedImageOSDiskSnapshotName: fakeOsDiskSnapshotName
`
result := artifact.String()
if result != expected {
t.Fatalf("bad: %s", result)
}
}
func TestArtifactString(t *testing.T) {
template := CaptureTemplate{
Resources: []CaptureResources{
@@ -126,7 +59,7 @@ func TestArtifactString(t *testing.T) {
},
}
artifact, err := NewArtifact(&template, getFakeSasUrl, "Linux")
artifact, err := NewArtifact(&template, getFakeSasUrl)
if err != nil {
t.Fatalf("err=%s", err)
}
@@ -147,9 +80,6 @@ func TestArtifactString(t *testing.T) {
if !strings.Contains(testSubject, "StorageAccountLocation: southcentralus") {
t.Errorf("Expected String() output to contain StorageAccountLocation")
}
if !strings.Contains(testSubject, "OSType: Linux") {
t.Errorf("Expected String() output to contain OSType")
}
}
func TestAdditionalDiskArtifactString(t *testing.T) {
@@ -177,7 +107,7 @@ func TestAdditionalDiskArtifactString(t *testing.T) {
},
}
artifact, err := NewArtifact(&template, getFakeSasUrl, "Linux")
artifact, err := NewArtifact(&template, getFakeSasUrl)
if err != nil {
t.Fatalf("err=%s", err)
}
@@ -198,9 +128,6 @@ func TestAdditionalDiskArtifactString(t *testing.T) {
if !strings.Contains(testSubject, "StorageAccountLocation: southcentralus") {
t.Errorf("Expected String() output to contain StorageAccountLocation")
}
if !strings.Contains(testSubject, "OSType: Linux") {
t.Errorf("Expected String() output to contain OSType")
}
if !strings.Contains(testSubject, "AdditionalDiskUri (datadisk-1): https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain AdditionalDiskUri")
}
@@ -227,7 +154,7 @@ func TestArtifactProperties(t *testing.T) {
},
}
testSubject, err := NewArtifact(&template, getFakeSasUrl, "Linux")
testSubject, err := NewArtifact(&template, getFakeSasUrl)
if err != nil {
t.Fatalf("err=%s", err)
}
@@ -247,9 +174,6 @@ func TestArtifactProperties(t *testing.T) {
if testSubject.StorageAccountLocation != "southcentralus" {
t.Errorf("Expected StorageAccountLocation to be 'southcentral', but got %s", testSubject.StorageAccountLocation)
}
if testSubject.OSType != "Linux" {
t.Errorf("Expected OSType to be 'Linux', but got %s", testSubject.OSType)
}
}
func TestAdditionalDiskArtifactProperties(t *testing.T) {
@@ -277,7 +201,7 @@ func TestAdditionalDiskArtifactProperties(t *testing.T) {
},
}
testSubject, err := NewArtifact(&template, getFakeSasUrl, "Linux")
testSubject, err := NewArtifact(&template, getFakeSasUrl)
if err != nil {
t.Fatalf("err=%s", err)
}
@@ -297,9 +221,6 @@ func TestAdditionalDiskArtifactProperties(t *testing.T) {
if testSubject.StorageAccountLocation != "southcentralus" {
t.Errorf("Expected StorageAccountLocation to be 'southcentral', but got %s", testSubject.StorageAccountLocation)
}
if testSubject.OSType != "Linux" {
t.Errorf("Expected OSType to be 'Linux', but got %s", testSubject.OSType)
}
if testSubject.AdditionalDisks == nil {
t.Errorf("Expected AdditionalDisks to be not nil")
}
@@ -332,7 +253,7 @@ func TestArtifactOverHyphenatedCaptureUri(t *testing.T) {
},
}
testSubject, err := NewArtifact(&template, getFakeSasUrl, "Linux")
testSubject, err := NewArtifact(&template, getFakeSasUrl)
if err != nil {
t.Fatalf("err=%s", err)
}
@@ -345,7 +266,7 @@ func TestArtifactOverHyphenatedCaptureUri(t *testing.T) {
func TestArtifactRejectMalformedTemplates(t *testing.T) {
template := CaptureTemplate{}
_, err := NewArtifact(&template, getFakeSasUrl, "Linux")
_, err := NewArtifact(&template, getFakeSasUrl)
if err == nil {
t.Fatalf("Expected artifact creation to fail, but it succeeded.")
}
@@ -368,7 +289,7 @@ func TestArtifactRejectMalformedStorageUri(t *testing.T) {
},
}
_, err := NewArtifact(&template, getFakeSasUrl, "Linux")
_, err := NewArtifact(&template, getFakeSasUrl)
if err == nil {
t.Fatalf("Expected artifact creation to fail, but it succeeded.")
}
+34 -3
View File
@@ -2,9 +2,40 @@ package arm
import (
"github.com/Azure/go-autorest/autorest/adal"
"github.com/Azure/go-autorest/autorest/azure"
)
type oAuthTokenProvider interface {
getServicePrincipalToken() (*adal.ServicePrincipalToken, error)
getServicePrincipalTokenWithResource(resource string) (*adal.ServicePrincipalToken, error)
type Authenticate struct {
env azure.Environment
clientID string
clientSecret string
tenantID string
}
func NewAuthenticate(env azure.Environment, clientID, clientSecret, tenantID string) *Authenticate {
return &Authenticate{
env: env,
clientID: clientID,
clientSecret: clientSecret,
tenantID: tenantID,
}
}
func (a *Authenticate) getServicePrincipalToken() (*adal.ServicePrincipalToken, error) {
return a.getServicePrincipalTokenWithResource(a.env.ResourceManagerEndpoint)
}
func (a *Authenticate) getServicePrincipalTokenWithResource(resource string) (*adal.ServicePrincipalToken, error) {
oauthConfig, err := adal.NewOAuthConfig(a.env.ActiveDirectoryEndpoint, a.tenantID)
if err != nil {
return nil, err
}
spt, err := adal.NewServicePrincipalToken(
*oauthConfig,
a.clientID,
a.clientSecret,
resource)
return spt, err
}
-155
View File
@@ -1,155 +0,0 @@
package arm
import (
"crypto/ecdsa"
"crypto/rand"
"crypto/rsa"
"crypto/sha1"
"crypto/x509"
"encoding/base64"
"encoding/pem"
"fmt"
"io/ioutil"
"os"
"time"
"github.com/Azure/go-autorest/autorest/azure"
jwt "github.com/dgrijalva/jwt-go"
)
func NewCertOAuthTokenProvider(env azure.Environment, clientID, clientCertPath, tenantID string) (oAuthTokenProvider, error) {
cert, key, err := readCert(clientCertPath)
if err != nil {
return nil, fmt.Errorf("Error reading certificate: %v", err)
}
audience := fmt.Sprintf("%s%s/oauth2/token", env.ActiveDirectoryEndpoint, tenantID)
jwt, err := makeJWT(clientID, audience, cert, key, time.Hour, true)
if err != nil {
return nil, fmt.Errorf("Error generating JWT: %v", err)
}
return NewJWTOAuthTokenProvider(env, clientID, jwt, tenantID), nil
}
// Creates a new JSON Web Token to be used as bearer JWT to authenticate
// to the Azure AD token endpoint to retrieve an access token for `audience`.
// If the full certificate is included in the token, then issuer/subject name
// could be used to authenticate if configured by the identity provider (AAD).
func makeJWT(clientID string, audience string,
cert *x509.Certificate, privatekey interface{},
validFor time.Duration, includeFullCertificate bool) (string, error) {
// The jti (JWT ID) claim provides a unique identifier for the JWT.
// See https://tools.ietf.org/html/rfc7519#section-4.1.7
jti := make([]byte, 20)
_, err := rand.Read(jti)
if err != nil {
return "", err
}
var token *jwt.Token
if cert.PublicKeyAlgorithm == x509.RSA {
token = jwt.New(jwt.SigningMethodRS256)
} else if cert.PublicKeyAlgorithm == x509.ECDSA {
token = jwt.New(jwt.SigningMethodES256)
} else {
return "", fmt.Errorf("Don't know how to handle this type of key algorithm: %v", cert.PublicKeyAlgorithm)
}
hasher := sha1.New()
if _, err := hasher.Write(cert.Raw); err != nil {
return "", err
}
thumbprint := base64.URLEncoding.EncodeToString(hasher.Sum(nil))
// X.509 thumbprint, see https://tools.ietf.org/html/rfc7515#section-4.1.7
token.Header["x5t"] = thumbprint
if includeFullCertificate {
// X.509 certificate (chain), see https://tools.ietf.org/html/rfc7515#section-4.1.6
token.Header["x5c"] = []string{base64.StdEncoding.EncodeToString(cert.Raw)}
}
token.Claims = jwt.MapClaims{
// See https://tools.ietf.org/html/rfc7519#section-4.1
"aud": audience,
"iss": clientID,
"sub": clientID,
"jti": base64.URLEncoding.EncodeToString(jti),
"nbf": time.Now().Unix(),
"exp": time.Now().Add(validFor).Unix(),
}
return token.SignedString(privatekey)
}
func readCert(file string) (cert *x509.Certificate, key interface{}, err error) {
f, err := os.Open(file)
if err != nil {
return nil, nil, err
}
defer f.Close()
d, err := ioutil.ReadAll(f)
if err != nil {
return nil, nil, err
}
blocks := []*pem.Block{}
for len(d) > 0 {
var b *pem.Block
b, d = pem.Decode(d)
if b == nil {
break
}
blocks = append(blocks, b)
}
certs := []*x509.Certificate{}
for _, block := range blocks {
if block.Type == "CERTIFICATE" {
c, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return nil, nil, fmt.Errorf(
"Failed to read certificate block: %v", err)
}
certs = append(certs, c)
} else if block.Type == "PRIVATE KEY" {
key, err = x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return nil, nil, fmt.Errorf(
"Failed to read private key block: %v", err)
}
}
// Don't care about other types of blocks, ignore
}
if key == nil {
return nil, nil, fmt.Errorf("Did not find private key in pem file")
}
// find the certificate that belongs to the private key by comparing the public keys
switch key := key.(type) {
case *rsa.PrivateKey:
for _, c := range certs {
if cp, ok := c.PublicKey.(*rsa.PublicKey); ok &&
(cp.N.Cmp(key.PublicKey.N) == 0) {
cert = c
}
}
case *ecdsa.PrivateKey:
for _, c := range certs {
if cp, ok := c.PublicKey.(*ecdsa.PublicKey); ok &&
(cp.X.Cmp(key.PublicKey.X) == 0) &&
(cp.Y.Cmp(key.PublicKey.Y) == 0) {
cert = c
}
}
}
if cert == nil {
return nil, nil, fmt.Errorf("Did not find certificate belonging to private key in pem file")
}
return cert, key, nil
}
@@ -1,36 +0,0 @@
package arm
import (
"fmt"
"strings"
"github.com/Azure/go-autorest/autorest/adal"
"github.com/Azure/go-autorest/autorest/azure"
packerAzureCommon "github.com/hashicorp/packer/builder/azure/common"
)
func NewDeviceFlowOAuthTokenProvider(env azure.Environment, say func(string), tenantID string) oAuthTokenProvider {
return &deviceflowOauthTokenProvider{}
}
type deviceflowOauthTokenProvider struct {
env azure.Environment
say func(string)
tenantID string
}
func (tp *deviceflowOauthTokenProvider) getServicePrincipalToken() (*adal.ServicePrincipalToken, error) {
return tp.getServicePrincipalTokenWithResource(tp.env.ResourceManagerEndpoint)
}
func (tp *deviceflowOauthTokenProvider) getServicePrincipalTokenWithResource(resource string) (*adal.ServicePrincipalToken, error) {
if resource == tp.env.ServiceManagementEndpoint {
tp.say("Getting auth token for Service management endpoint")
} else if resource == strings.TrimRight(tp.env.KeyVaultEndpoint, "/") {
tp.say("Getting token for Vault resource")
} else {
tp.say(fmt.Sprintf("Getting token for %s", resource))
}
return packerAzureCommon.Authenticate(tp.env, tp.tenantID, tp.say, resource)
}

Some files were not shown because too many files have changed in this diff Show More