Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: Can't use user variable in keep_input_artifact #2630

Closed
styro opened this issue Aug 19, 2015 · 46 comments
Closed

core: Can't use user variable in keep_input_artifact #2630

styro opened this issue Aug 19, 2015 · 46 comments
Assignees

Comments

@styro
Copy link

styro commented Aug 19, 2015

I've got the same issue as #1913 recurring in 0.8.5 when using a string user variable for the boolean keep_input_artifact in the vagrant post-processor.

Oddly enough though, I can use the same string syntax for booleans successfully in the headless setting in the virtualbox builder.

@cbednarski
Copy link
Contributor

Thanks for the report. I've seen some options that want a boolean and others that want a string "true" or "false". I agree this is confusing and we should probably accept either one in any of these cases.

This particular case might be caused by the interpolation happening after the value in the template has been coerced into the appropriate type, so we probably need to coerce the interpolated value as well. I think this usually works as expected so I'm not sure what's different in this case.

@styro
Copy link
Author

styro commented Aug 21, 2015

I personally don't mind whether it's a string or a boolean, as long as at least one of them works :)

@dragon788
Copy link
Contributor

Having this issue with 0.8.6 as well. Was just trying to be 'clever' and handle keeping the artifact based on whether I was in "dev" mode or "ci" mode and it appears I currently cannot do this due to the strict typing on keep_input_artifact.

@BlinkyStitt
Copy link

I'm running into this with 0.9.0 so I'm stuck on 0.7.5

@ferrarimarco
Copy link

ferrarimarco commented Nov 28, 2016

I tried with packer 0.12.0 and this issue is still valid (discovered while working on boxcutter/ubuntu#87). Here is the log:

$ packer build ubuntu.json
Failed to parse template: 1 error(s) occurred:

* post-processor 1.1: 1 error(s) decoding:

* 'keep_input_artifact' expected type 'bool', got unconvertible type 'string'

Here are the relevant parts of the template:

  "variables": {
    "keep_input_artifact": "true",
  }
  "post-processors": [
    {
      "keep_input_artifact": "{{ user `keep_input_artifact` }}",
      "output": "ubuntu.box",
      "type": "vagrant",
    }

I am using boxcutter/ubuntu 16.04 template.

I don't know if this issue is still observed. Should I open a new one? Thanks ping @cbednarski

@rickard-von-essen
Copy link
Collaborator

@ferrarimarco Gist full log (PACKER_LOG=1 packer build template.json)

@ferrarimarco
Copy link

ferrarimarco commented Nov 28, 2016

Here it is, @rickard-von-essen thanks:

$ PACKER_LOG=1 packer build -only=virtualbox-iso -var-file=ubuntu1604.json ubuntu.json
2016/11/28 12:32:31 [INFO] Packer version: 0.12.0
2016/11/28 12:32:31 Packer Target OS/Arch: windows amd64
2016/11/28 12:32:31 Built with Go Version: go1.7.3
2016/11/28 12:32:31 Using internal plugin for qemu
2016/11/28 12:32:31 Using internal plugin for vmware-iso
2016/11/28 12:32:31 Using internal plugin for cloudstack
2016/11/28 12:32:31 Using internal plugin for googlecompute
2016/11/28 12:32:31 Using internal plugin for openstack
2016/11/28 12:32:31 Using internal plugin for file
2016/11/28 12:32:31 Using internal plugin for null
2016/11/28 12:32:31 Using internal plugin for amazon-ebs
2016/11/28 12:32:31 Using internal plugin for amazon-ebsvolume
2016/11/28 12:32:31 Using internal plugin for azure-arm
2016/11/28 12:32:31 Using internal plugin for parallels-pvm
2016/11/28 12:32:31 Using internal plugin for profitbricks
2016/11/28 12:32:31 Using internal plugin for virtualbox-iso
2016/11/28 12:32:31 Using internal plugin for virtualbox-ovf
2016/11/28 12:32:31 Using internal plugin for amazon-instance
2016/11/28 12:32:31 Using internal plugin for digitalocean
2016/11/28 12:32:31 Using internal plugin for parallels-iso
Failed to parse template: 1 error(s) occurred:
2016/11/28 12:32:31 Using internal plugin for amazon-chroot
2016/11/28 12:32:31 Using internal plugin for docker
2016/11/28 12:32:31 Using internal plugin for vmware-vmx

2016/11/28 12:32:31 Using internal plugin for windows-restart
2016/11/28 12:32:31 Using internal plugin for ansible
2016/11/28 12:32:31 Using internal plugin for chef-client
* post-processor 1.1: 1 error(s) decoding:
2016/11/28 12:32:31 Using internal plugin for chef-solo
2016/11/28 12:32:31 Using internal plugin for file
2016/11/28 12:32:31 Using internal plugin for puppet-server

2016/11/28 12:32:31 Using internal plugin for shell-local
2016/11/28 12:32:31 Using internal plugin for windows-shell
2016/11/28 12:32:31 Using internal plugin for ansible-local
2016/11/28 12:32:31 Using internal plugin for powershell
* 'keep_input_artifact' expected type 'bool', got unconvertible type 'string'
2016/11/28 12:32:31 Using internal plugin for puppet-masterless
2016/11/28 12:32:31 Using internal plugin for salt-masterless
2016/11/28 12:32:31 Using internal plugin for shell
2016/11/28 12:32:31 Using internal plugin for artifice
2016/11/28 12:32:31 Using internal plugin for docker-tag
2016/11/28 12:32:31 Using internal plugin for manifest
2016/11/28 12:32:31 Using internal plugin for amazon-import
2016/11/28 12:32:31 Using internal plugin for docker-import
2016/11/28 12:32:31 Using internal plugin for googlecompute-export
2016/11/28 12:32:31 Using internal plugin for vagrant-cloud
2016/11/28 12:32:31 Using internal plugin for atlas
2016/11/28 12:32:31 Using internal plugin for checksum
2016/11/28 12:32:31 Using internal plugin for compress
2016/11/28 12:32:31 Using internal plugin for docker-push
2016/11/28 12:32:31 Using internal plugin for shell-local
2016/11/28 12:32:31 Using internal plugin for vagrant
2016/11/28 12:32:31 Using internal plugin for docker-save
2016/11/28 12:32:31 Using internal plugin for vsphere
2016/11/28 12:32:31 Attempting to open config file: C:\Users\dev-ferrari\AppData\Roaming\packer.config
2016/11/28 12:32:31 [WARN] Config file doesn't exist: C:\Users\dev-ferrari\AppData\Roaming\packer.config
2016/11/28 12:32:31 Packer config: &{DisableCheckpoint:false DisableCheckpointSignature:false PluginMinPort:10000 PluginMaxPort:25000 Builders:map[amazon-chroot:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-amazon-chroot vmware-vmx:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-vmware-vmx null:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-null amazon-ebsvolume:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-amazon-ebsvolume virtualbox-iso:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-virtualbox-iso virtualbox-ovf:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-virtualbox-ovf amazon-instance:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-amazon-instance cloudstack:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-cloudstack parallels-pvm:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-parallels-pvm profitbricks:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-profitbricks docker:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-docker qemu:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-qemu openstack:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-openstack file:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-file amazon-ebs:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-amazon-ebs digitalocean:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-digitalocean vmware-iso:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-vmware-iso googlecompute:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-googlecompute azure-arm:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-azure-arm parallels-iso:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-builder-parallels-iso] PostProcessors:map[shell-local:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-shell-local vagrant:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-vagrant artifice:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-artifice amazon-import:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-amazon-import checksum:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-checksum docker-push:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-docker-push docker-import:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-docker-import vagrant-cloud:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-vagrant-cloud atlas:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-atlas compress:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-compress docker-save:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-docker-save vsphere:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-vsphere docker-tag:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-docker-tag manifest:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-manifest googlecompute-export:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-post-processor-googlecompute-export] Provisioners:map[file:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-file shell-local:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-shell-local windows-restart:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-windows-restart chef-client:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-chef-client chef-solo:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-chef-solo ansible-local:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-ansible-local powershell:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-powershell puppet-masterless:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-puppet-masterless salt-masterless:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-salt-masterless shell:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-shell ansible:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-ansible puppet-server:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-puppet-server windows-shell:C:\Program Files\packer\packer.exe-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-windows-shell]}
2016/11/28 12:32:31 Setting cache directory: C:\Users\ferrarim\Workspaces\ubuntu\packer_cache
2016/11/28 12:32:31 ui error: Failed to parse template: 1 error(s) occurred:

* post-processor 1.1: 1 error(s) decoding:

* 'keep_input_artifact' expected type 'bool', got unconvertible type 'string'
2016/11/28 12:32:31 waiting for all plugin processes to complete...

@ferrarimarco
Copy link

@rickard-von-essen I tried with the latest devevelopment version, no changes:

/opt # packer version
Packer v0.12.1.dev (e45dd0b3949f16ddc1447f25030b3fc69cd7a137)

/opt # packer validate ubuntu.json
Failed to parse template: 1 error(s) occurred:

* post-processor 1.1: 1 error(s) decoding:

* 'keep_input_artifact' expected type 'bool', got unconvertible type 'string'

@rickard-von-essen
Copy link
Collaborator

$ packer version
Packer v0.12.1.dev (17eda06d613aa18ae1da05c80800d5070b19ee11+CHANGES)

$ packer validate ubuntu.json
Template validated successfully.

Could you try on another system or in a vagrant box etc?

@ferrarimarco
Copy link

That log was generated from inside a Docker container, starting from hashicorp/docker-hub-images Dockerfile-full.

I just removed the ENTRYPOINT instruction because otherwise the container could not start at all.

@rickard-von-essen
Copy link
Collaborator

Ok, do you access the ubuntu repo through a volume mount into docker?

@ferrarimarco
Copy link

I tried in two ways:

  1. Volume mount
  2. Clone the repository directly in the container with Git, to exclude any mount side effects

@rickard-von-essen
Copy link
Collaborator

So what is the exact output of first:

docker run -it ubuntu:trusty bash

and then:

apt-get update > /dev/null && apt-get -y install git unzip wget > /dev/null

wget https://releases.hashicorp.com/packer/0.12.0/packer_0.12.0_linux_amd64.zip
unzip packer_0.12.0_linux_amd64.zip

git clone https://github.com/boxcutter/ubuntu.git && cd ubuntu && ../packer validate ubuntu.json

@ferrarimarco
Copy link

Here are the results with the "vanilla" template.

root@8ca1443a6c62:/# apt-get update > /dev/null && apt-get -y install git unzip wget > /dev/null
root@8ca1443a6c62:/# wget https://releases.hashicorp.com/packer/0.12.0/packer_0.12.0_linux_amd64.zip
--2016-11-30 07:58:17--  https://releases.hashicorp.com/packer/0.12.0/packer_0.12.0_linux_amd64.zip
Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.120.69
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.120.69|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10047746 (9.6M) [application/zip]
Saving to: 'packer_0.12.0_linux_amd64.zip'

100%[====================================================================================================================================================>] 10,047,746   777KB/s   in 9.5s   

2016-11-30 07:58:29 (1.01 MB/s) - 'packer_0.12.0_linux_amd64.zip' saved [10047746/10047746]

root@8ca1443a6c62:/# unzip packer_0.12.0_linux_amd64.zip  
Archive:  packer_0.12.0_linux_amd64.zip
  inflating: packer                  
root@8ca1443a6c62:/# git clone https://github.com/boxcutter/ubuntu.git && cd ubuntu && ../packer validate ubuntu.json
Cloning into 'ubuntu'...
remote: Counting objects: 2065, done.
remote: Compressing objects: 100% (44/44), done.
remote: Total 2065 (delta 18), reused 0 (delta 0), pack-reused 2019
Receiving objects: 100% (2065/2065), 372.84 KiB | 0 bytes/s, done.
Resolving deltas: 100% (1345/1345), done.
Checking connectivity... done.
Template validated successfully.

My edits to ubuntu.json:

  "variables": {
    "keep_input_artifact": "true",
  }
...
  "post-processors": [
    {
      "keep_input_artifact": "{{ user `keep_input_artifact` }}",
      ...
    }

Results:

root@8ca1443a6c62:/ubuntu# ../packer validate ubuntu.json
Failed to parse template: 1 error(s) occurred:

* post-processor 1.1: 1 error(s) decoding:

* 'keep_input_artifact' expected type 'bool', got unconvertible type 'string'
root@8ca1443a6c62:/ubuntu# 

Thanks @rickard-von-essen

@rickard-von-essen
Copy link
Collaborator

Check! I got a bit lost while trying to understand this.

I think a hint on why this doesn't work is that keep_input_artifact is the only bool which doesn't come from the plugins. It's defined in template/template.go.

@SwampDragons SwampDragons added config-2.0 wontfix Out of scope/alignment with the project, or issue is expected, intended behavior and removed post-processor/vagrant labels Jan 25, 2019
@SwampDragons
Copy link
Contributor

Hopefully we'll be able to fix this when we implement the HCL2 configs; I'm not going to prioritize it before then because it's got a workaround. To paraphrase Matthew's comment above, It's a deep architectural quirk and the refactor necessary to make this one key work is dramatically more effort than implementing a workaround on your end. For now I'm going to mark this with the "config-2.0" and "wontfix" flags to hopefully remove confusion about the maintainers' stance on this bug.

tl;dr We know it's still kickin' around and we aren't going to fix it independently of the large refactor we already have planned for late 2019.

@mschuchard
Copy link

I thought the HCL2 configs were already in Packer 1.3.x, since they were a prereq for the Vault integrations that @SwampDragons coincidentally implemented.

Additionally, will the HCL2 configs enable Arrays as well instead of the e.g. "us-east-1,us-east-2" workaround?

@SwampDragons
Copy link
Contributor

HCL2 turned out not to be necessary for that Vault integration.

@SwampDragons
Copy link
Contributor

I think whether we support arrays for particular options will have to do with the option in question.

@azr
Copy link
Contributor

azr commented Jul 29, 2020

Hello there, just to say that this is now possible to set keep_input_artifact using variables in HCL2 ! I'm going to go ahead and close that issue ! Thanks !

@azr azr closed this as completed Jul 29, 2020
@shibumi
Copy link

shibumi commented Jul 29, 2020

@azr can you give an example config file for visualizing how to do this?

@azr
Copy link
Contributor

azr commented Jul 30, 2020

Of course, one thing we can do is use an input variable:

variable "keep_that_input_artifact" {
 default = true
}

build {
  source = ["..."] # your sources
  
  # provision ...
  
  post-process "shell-local" {
    keep_input_artifact = var.keep_that_input_artifact
  }
}

That variable could be set from cli arguments; env or a config file; .

@SwampDragons SwampDragons reopened this Jul 30, 2020
@azr
Copy link
Contributor

azr commented Jul 30, 2020

Ah no, wait; @SwampDragons just found out that I was saying something wrong; the 'eval context' is not present at the moment we parse these variables and therefore we cannot use variables there het;

Issue is here: the second argument of decodebody should just take it as an argument:

func (p *Parser) decodePostProcessor(block *hcl.Block) (*PostProcessorBlock, hcl.Diagnostics) {
var b struct {
Name string `hcl:"name,optional"`
Only []string `hcl:"only,optional"`
Except []string `hcl:"except,optional"`
KeepInputArtifact *bool `hcl:"keep_input_artifact,optional"`
Rest hcl.Body `hcl:",remain"`
}
diags := gohcl.DecodeBody(block.Body, nil, &b)
if diags.HasErrors() {
return nil, diags
}

@SwampDragons SwampDragons added this to the 1.7.1 milestone Feb 19, 2021
@SwampDragons SwampDragons added hcl2 and removed config-2.0 wontfix Out of scope/alignment with the project, or issue is expected, intended behavior labels Feb 19, 2021
@nywilken nywilken modified the milestones: 1.7.1, 1.7.2 Mar 30, 2021
@sylviamoss sylviamoss modified the milestones: 1.7.2, 1.7.3 Apr 6, 2021
@nywilken nywilken modified the milestones: 1.7.3, 1.7.7 Oct 13, 2021
@nywilken nywilken self-assigned this Nov 30, 2021
@nywilken
Copy link
Contributor

Support for using variable interpolation in post-processors was add in the 1.7.3 release via the change #11094. I'm going to close this issue, as it can no longer be reproduced. If you are still running into issues please drop us a comment and we will gladly reopen. Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests