Merge pull request #556 from runatlantis/shell-format

New version 3 of atlantis.yaml
This commit is contained in:
Luke Kysow
2019-03-28 15:02:06 -05:00
committed by GitHub
32 changed files with 466 additions and 156 deletions

View File

@@ -808,7 +808,7 @@ when new commits are pushed to the pull request.
## Backwards Incompatibilities / Notes:
- The old `atlantis.yaml` config file format is not supported. You will need to migrate to the new config
format, see: https://www.runatlantis.io/docs/upgrading-atlantis-yaml-to-version-2.html
format, see: https://www.runatlantis.io/docs/upgrading-atlantis-yaml.html
- To use the new config file, you must run Atlantis with `--allow-repo-config`.
- Atlantis will now try to automatically plan. To disable this, you'll need to create an `atlantis.yaml` file
as follows:

View File

@@ -5,7 +5,7 @@
<img v-if="data.heroImage" :src="$withBase(data.heroImage)" alt="hero">
<h1>{{ data.heroText || $title || 'Hello' }}</h1>
<p class="description">
Start working on Terraform <strong>as a team</strong>.
Terraform Pull Request Automation
</p>
<p class="action" v-if="data.actionText && data.actionLink">
<a href="/guide/" class="nav-link action-button">Get Started </a>

View File

@@ -62,7 +62,7 @@ module.exports = {
'server-side-repo-config',
'custom-workflows',
'repo-level-atlantis-yaml',
'upgrading-atlantis-yaml-to-version-2',
'upgrading-atlantis-yaml',
'apply-requirements',
'checkout-strategy',
'terraform-versions',

View File

@@ -36,7 +36,7 @@ You can set the `approved` requirement by:
#### atlantis.yaml
```yaml
version: 2
version: 3
projects:
- dir: .
apply_requirements: [approved]
@@ -78,7 +78,7 @@ You can set the `mergeable` requirement by:
#### atlantis.yaml
```yaml
version: 2
version: 3
projects:
- dir: .
apply_requirements: [mergeable]
@@ -163,7 +163,7 @@ If you only want some projects/repos to have apply requirements, then you must
#### atlatis.yaml
```yaml
version: 2
version: 3
projects:
- dir: staging
# By default, apply_requirements is empty so this

View File

@@ -11,7 +11,7 @@ Automerging can be enabled either by:
pull requests to be automerged and any repo config will be ignored.
1. Setting `automerge: true` in the repo's `atlantis.yaml` file:
```yaml
version: 2
version: 3
automerge: true
projects:
- dir: .

View File

@@ -51,7 +51,7 @@ workflows:
Then in your repo-level `atlantis.yaml` file, you would reference the workflows:
```yaml
# atlantis.yaml
version: 2
version: 3
projects:
# If two or more projects have the same dir and workspace, they must also have
# a 'name' key to differentiate them.
@@ -163,7 +163,7 @@ workflows:
Which you would then reference in your repo-level `atlantis.yaml`:
```yaml
version: 2
version: 3
projects:
- dir: live/staging
workflow: terragrunt
@@ -233,7 +233,7 @@ will complain in-between commands since the backend config has changed.
You would then reference the workflows in your repo-level `atlantis.yaml`:
```yaml
version: 2
version: 3
projects:
- name: staging
dir: .

View File

@@ -44,7 +44,7 @@ need to be defined.
## Example Using All Keys
```yaml
version: 2
version: 3
automerge: true
projects:
- name: my-project-name
@@ -74,7 +74,7 @@ workflows:
## Use Cases
### Disabling Autoplanning
```yaml
version: 2
version: 3
projects:
- dir: project1
autoplan:
@@ -101,7 +101,7 @@ If you wanted Atlantis to autoplan `project1/` whenever any `.tf` file under `mo
changed or any `.tf` or `.tfvars` file under `project1/` changed, you could use the following configuration:
```yaml
version: 2
version: 3
projects:
- dir: project1
autoplan:
@@ -113,7 +113,7 @@ Note:
### Supporting Terraform Workspaces
```yaml
version: 2
version: 3
projects:
- dir: project1
workspace: staging
@@ -152,7 +152,7 @@ If you'd like to use a different version of Terraform than what is in Atlantis'
`PATH` or is set by the `--default-tf-version` flag, then set the `terraform_version` key:
```yaml
version: 2
version: 3
projects:
- dir: project1
terraform_version: 0.10.0
@@ -163,7 +163,7 @@ Atlantis will automatically download and use this version.
### Requiring Approvals For Production
In this example, we only want to require `apply` approvals for the `production` directory.
```yaml
version: 2
version: 3
projects:
- dir: staging
- dir: production

View File

@@ -143,7 +143,7 @@ Then each allowed repo can have an `atlantis.yaml` file that
sets `apply_requirements` to an empty array (disabling the requirement).
```yaml
# atlantis.yaml in the repo root
version: 2
version: 3
projects:
- dir: .
apply_requirements: []
@@ -210,7 +210,7 @@ files:
```yaml
# atlantis.yaml
version: 2
version: 3
projects:
- dir: .
workflow: custom1 # could also be custom2 OR default
@@ -250,7 +250,7 @@ repos:
Then each allowed repo can define and use a custom workflow in their `atlantis.yaml` files:
```yaml
# atlantis.yaml
version: 2
version: 3
projects:
- dir: .
workflow: custom1

View File

@@ -6,7 +6,7 @@ the `--default-tf-version` flag (ex. `--default-tf-version=v0.12.0`).
If you wish to use a different version than the default for a specific repo or project, you need
to create an `atlantis.yaml` file and set the `terraform_version` key:
```yaml
version: 2
version: 3
projects:
- dir: .
terraform_version: v0.10.5

View File

@@ -1,8 +1,45 @@
# Upgrading atlantis.yaml To Version 2
These docs describe how to upgrade your `atlantis.yaml` file from the format used
in Atlantis `<=v0.3.10`.
# Upgrading atlantis.yaml
## Single atlantis.yaml
## Upgrading From v2 To v3
Atlantis version `v0.7.0` introduced a new version 3 of `atlantis.yaml`.
**If you're not using [custom `run` steps](custom-workflows.html#custom-run-command),
then you can upgrade from `version: 2` to `version: 3` without any changes.**
**NOTE:** Version 2 **is not being deprecated** and there is no need to upgrade your version
if you don't wish to do so.
The only change from v2 to v3 is that we're parsing custom `run` steps differently.
```yaml
# atlantis.yaml
worklows:
custom:
plan:
steps:
- run: my custom command
```
<center><i>An example workflow using a custom run step</i></center>
Previously, we used a library that would parse the custom step prior to running
it. Now, we just run the step directly. This will only affect your steps if they were using shell escaping of some sort.
For example, if your step was previously:
```yaml
# version: 2
- run: "printf \'print me\'"
```
You can now write this in version 3 as:
```yaml
# version: 3
- run: "printf 'print me'"
```
## Upgrading From V1 To V3
If you are upgrading from an **old** Atlantis version `<=v0.3.10` (from before July 4, 2018)
you'll need to follow the following steps.
### Single atlantis.yaml
If you had multiple `atlantis.yaml` files per directory then you'll need to
consolidate them into a single `atlantis.yaml` file at the root of the repo.
@@ -42,7 +79,7 @@ workflows:
We will talk more about `workflows` below.
## Terraform Version
### Terraform Version
The `terraform_version` key moved from being a top-level key to being per `project`
so if before your `atlantis.yaml` was in directory `mydir` and looked like:
```yaml
@@ -57,7 +94,7 @@ projects:
terraform_version: 0.11.0
```
## Workflows
### Workflows
Workflows are the new way to set all `pre_*`, `post_*` and `extra_arguments`.
Each `project` can have a custom workflow via the `workflow` key.
@@ -105,7 +142,7 @@ workflows:
...
```
### Extra Arguments
#### Extra Arguments
`extra_arguments` is now specified as follows. Given a previous config:
```yaml
extra_arguments:
@@ -138,7 +175,7 @@ workflows:
```
### Pre/Post Commands
#### Pre/Post Commands
Instead of using `pre_*` or `post_*`, you now can insert your custom commands
before/after the built-in commands. Given a previous config:

View File

@@ -0,0 +1,117 @@
// Code generated by pegomock. DO NOT EDIT.
// Source: github.com/runatlantis/atlantis/server/events (interfaces: CustomStepRunner)
package mocks
import (
pegomock "github.com/petergtz/pegomock"
models "github.com/runatlantis/atlantis/server/events/models"
"reflect"
"time"
)
type MockCustomStepRunner struct {
fail func(message string, callerSkip ...int)
}
func NewMockCustomStepRunner(options ...pegomock.Option) *MockCustomStepRunner {
mock := &MockCustomStepRunner{}
for _, option := range options {
option.Apply(mock)
}
return mock
}
func (mock *MockCustomStepRunner) SetFailHandler(fh pegomock.FailHandler) { mock.fail = fh }
func (mock *MockCustomStepRunner) FailHandler() pegomock.FailHandler { return mock.fail }
func (mock *MockCustomStepRunner) Run(ctx models.ProjectCommandContext, cmd string, path string) (string, error) {
if mock == nil {
panic("mock must not be nil. Use myMock := NewMockCustomStepRunner().")
}
params := []pegomock.Param{ctx, cmd, path}
result := pegomock.GetGenericMockFrom(mock).Invoke("Run", params, []reflect.Type{reflect.TypeOf((*string)(nil)).Elem(), reflect.TypeOf((*error)(nil)).Elem()})
var ret0 string
var ret1 error
if len(result) != 0 {
if result[0] != nil {
ret0 = result[0].(string)
}
if result[1] != nil {
ret1 = result[1].(error)
}
}
return ret0, ret1
}
func (mock *MockCustomStepRunner) VerifyWasCalledOnce() *VerifierCustomStepRunner {
return &VerifierCustomStepRunner{
mock: mock,
invocationCountMatcher: pegomock.Times(1),
}
}
func (mock *MockCustomStepRunner) VerifyWasCalled(invocationCountMatcher pegomock.Matcher) *VerifierCustomStepRunner {
return &VerifierCustomStepRunner{
mock: mock,
invocationCountMatcher: invocationCountMatcher,
}
}
func (mock *MockCustomStepRunner) VerifyWasCalledInOrder(invocationCountMatcher pegomock.Matcher, inOrderContext *pegomock.InOrderContext) *VerifierCustomStepRunner {
return &VerifierCustomStepRunner{
mock: mock,
invocationCountMatcher: invocationCountMatcher,
inOrderContext: inOrderContext,
}
}
func (mock *MockCustomStepRunner) VerifyWasCalledEventually(invocationCountMatcher pegomock.Matcher, timeout time.Duration) *VerifierCustomStepRunner {
return &VerifierCustomStepRunner{
mock: mock,
invocationCountMatcher: invocationCountMatcher,
timeout: timeout,
}
}
type VerifierCustomStepRunner struct {
mock *MockCustomStepRunner
invocationCountMatcher pegomock.Matcher
inOrderContext *pegomock.InOrderContext
timeout time.Duration
}
func (verifier *VerifierCustomStepRunner) Run(ctx models.ProjectCommandContext, cmd string, path string) *CustomStepRunner_Run_OngoingVerification {
params := []pegomock.Param{ctx, cmd, path}
methodInvocations := pegomock.GetGenericMockFrom(verifier.mock).Verify(verifier.inOrderContext, verifier.invocationCountMatcher, "Run", params, verifier.timeout)
return &CustomStepRunner_Run_OngoingVerification{mock: verifier.mock, methodInvocations: methodInvocations}
}
type CustomStepRunner_Run_OngoingVerification struct {
mock *MockCustomStepRunner
methodInvocations []pegomock.MethodInvocation
}
func (c *CustomStepRunner_Run_OngoingVerification) GetCapturedArguments() (models.ProjectCommandContext, string, string) {
ctx, cmd, path := c.GetAllCapturedArguments()
return ctx[len(ctx)-1], cmd[len(cmd)-1], path[len(path)-1]
}
func (c *CustomStepRunner_Run_OngoingVerification) GetAllCapturedArguments() (_param0 []models.ProjectCommandContext, _param1 []string, _param2 []string) {
params := pegomock.GetGenericMockFrom(c.mock).GetInvocationParams(c.methodInvocations)
if len(params) > 0 {
_param0 = make([]models.ProjectCommandContext, len(params[0]))
for u, param := range params[0] {
_param0[u] = param.(models.ProjectCommandContext)
}
_param1 = make([]string, len(params[1]))
for u, param := range params[1] {
_param1[u] = param.(string)
}
_param2 = make([]string, len(params[2]))
for u, param := range params[2] {
_param2[u] = param.(string)
}
}
return
}

View File

@@ -317,6 +317,9 @@ type ProjectCommandContext struct {
// ProjectName is the name of the project set in atlantis.yaml. If there was
// no name this will be an empty string.
ProjectName string
// RepoConfigVersion is the version of the repo's atlantis.yaml file. If
// there was no file, this will be 0.
RepoConfigVersion int
// RePlanCmd is the command that users should run to re-plan this project.
// If this is an apply then this will be empty.
RePlanCmd string

View File

@@ -394,6 +394,7 @@ func (p *DefaultProjectCommandBuilder) buildCtx(ctx *CommandContext,
ApplyRequirements: projCfg.ApplyRequirements,
RePlanCmd: p.CommentBuilder.BuildPlanComment(projCfg.RepoRelDir, projCfg.Workspace, projCfg.Name, commentArgs),
RepoRelDir: projCfg.RepoRelDir,
RepoConfigVersion: projCfg.RepoCfgVersion,
TerraformVersion: projCfg.TerraformVersion,
User: ctx.User,
Verbose: verbose,

View File

@@ -87,7 +87,7 @@ workflows:
steps:
- apply`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -109,6 +109,7 @@ projects:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
TerraformVersion: mustVersion("10.0"),
@@ -137,7 +138,7 @@ workflows:
steps:
- apply`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -159,6 +160,7 @@ projects:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{"approved", "mergeable"},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
TerraformVersion: mustVersion("10.0"),
@@ -195,7 +197,7 @@ workflows:
apply:
steps: []`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -217,6 +219,7 @@ projects:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{"approved"},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
TerraformVersion: mustVersion("10.0"),
@@ -246,7 +249,7 @@ workflows:
steps:
- apply`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -273,7 +276,7 @@ workflows:
steps:
- apply`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -301,7 +304,7 @@ workflows:
steps:
- apply`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -330,7 +333,7 @@ workflows:
steps: []
`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -362,6 +365,7 @@ workflows:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
TerraformVersion: mustVersion("10.0"),
@@ -393,7 +397,7 @@ workflows:
steps: [apply]
`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -416,6 +420,7 @@ projects:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
TerraformVersion: mustVersion("10.0"),
@@ -444,7 +449,7 @@ workflows:
steps: [apply]
`,
repoCfg: `
version: 2
version: 3
automerge: true
projects:
- dir: project1
@@ -473,6 +478,7 @@ workflows:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
TerraformVersion: mustVersion("10.0"),
@@ -497,7 +503,7 @@ workflows:
steps: [plan]
`,
repoCfg: `
version: 2
version: 3
projects:
- dir: project1
workspace: myworkspace
@@ -514,6 +520,7 @@ projects:
Pull: models.PullRequest{},
ProjectName: "",
ApplyRequirements: []string{"approved"},
RepoConfigVersion: 3,
RePlanCmd: "atlantis plan -d project1 -w myworkspace -- flag",
RepoRelDir: "project1",
User: models.User{},

View File

@@ -35,7 +35,7 @@ func TestDefaultProjectCommandBuilder_BuildAutoplanCommands(t *testing.T) {
{
Description: "simple atlantis.yaml",
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: .
`,
@@ -50,7 +50,7 @@ projects:
{
Description: "some projects disabled",
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: .
autoplan:
@@ -79,7 +79,7 @@ projects:
{
Description: "some projects disabled",
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: .
autoplan:
@@ -107,7 +107,7 @@ projects:
{
Description: "no projects modified",
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: mydir
`,
@@ -203,7 +203,7 @@ func TestDefaultProjectCommandBuilder_BuildSinglePlanApplyCommand(t *testing.T)
Workspace: "myworkspace",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: .
workspace: myworkspace
@@ -220,7 +220,7 @@ projects:
Workspace: "myworkspace",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: notroot
workspace: myworkspace
@@ -237,7 +237,7 @@ projects:
Workspace: "myworkspace",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: .
workspace: notmyworkspace
@@ -251,7 +251,7 @@ projects:
ProjectName: "myproject",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- name: myproject
dir: .
@@ -269,7 +269,7 @@ projects:
ProjectName: "myproject",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- name: myproject
dir: .
@@ -287,7 +287,7 @@ projects:
ProjectName: "myproject",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- name: myproject
dir: .
@@ -306,7 +306,7 @@ projects:
Workspace: "myworkspace",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- name: myproject
dir: .
@@ -327,7 +327,7 @@ projects:
ProjectName: "notconfigured",
},
AtlantisYAML: `
version: 2
version: 3
projects:
- dir: .
`,
@@ -447,7 +447,7 @@ func TestDefaultProjectCommandBuilder_BuildPlanCommands(t *testing.T) {
"main.tf": nil,
},
},
AtlantisYAML: `version: 2
AtlantisYAML: `version: 3
projects:
- dir: project1 # project1 uses the defaults
- dir: project2 # project2 has autoplan disabled but should use default when_modified
@@ -606,7 +606,7 @@ func TestDefaultProjectCommandBuilder_WrongWorkspaceName(t *testing.T) {
defer cleanup()
repoDir := filepath.Join(tmpDir, "pulldir/notconfigured")
yamlCfg := `version: 2
yamlCfg := `version: 3
projects:
- dir: .
workspace: default

View File

@@ -55,6 +55,14 @@ type StepRunner interface {
Run(ctx models.ProjectCommandContext, extraArgs []string, path string) (string, error)
}
//go:generate pegomock generate -m --use-experimental-model-gen --package mocks -o mocks/mock_custom_step_runner.go CustomStepRunner
// CustomStepRunner runs custom run steps.
type CustomStepRunner interface {
// Run cmd in path.
Run(ctx models.ProjectCommandContext, cmd string, path string) (string, error)
}
//go:generate pegomock generate -m --use-experimental-model-gen --package mocks -o mocks/mock_webhooks_sender.go WebhooksSender
// WebhooksSender sends webhook.
@@ -81,7 +89,7 @@ type DefaultProjectCommandRunner struct {
InitStepRunner StepRunner
PlanStepRunner StepRunner
ApplyStepRunner StepRunner
RunStepRunner StepRunner
RunStepRunner CustomStepRunner
PullApprovedChecker runtime.PullApprovedChecker
WorkingDir WorkingDir
Webhooks WebhooksSender

View File

@@ -34,7 +34,7 @@ func TestDefaultProjectCommandRunner_Plan(t *testing.T) {
mockInit := mocks.NewMockStepRunner()
mockPlan := mocks.NewMockStepRunner()
mockApply := mocks.NewMockStepRunner()
mockRun := mocks.NewMockStepRunner()
mockRun := mocks.NewMockCustomStepRunner()
mockWorkingDir := mocks.NewMockWorkingDir()
mockLocker := mocks.NewMockProjectLocker()
@@ -94,7 +94,7 @@ func TestDefaultProjectCommandRunner_Plan(t *testing.T) {
When(mockInit.Run(ctx, nil, repoDir)).ThenReturn("init", nil)
When(mockPlan.Run(ctx, nil, repoDir)).ThenReturn("plan", nil)
When(mockApply.Run(ctx, nil, repoDir)).ThenReturn("apply", nil)
When(mockRun.Run(ctx, nil, repoDir)).ThenReturn("run", nil)
When(mockRun.Run(ctx, "", repoDir)).ThenReturn("run", nil)
res := runner.Plan(ctx)
@@ -112,7 +112,7 @@ func TestDefaultProjectCommandRunner_Plan(t *testing.T) {
case "apply":
mockApply.VerifyWasCalledOnce().Run(ctx, nil, repoDir)
case "run":
mockRun.VerifyWasCalledOnce().Run(ctx, nil, repoDir)
mockRun.VerifyWasCalledOnce().Run(ctx, "", repoDir)
}
}
}
@@ -250,7 +250,7 @@ func TestDefaultProjectCommandRunner_Apply(t *testing.T) {
mockInit := mocks.NewMockStepRunner()
mockPlan := mocks.NewMockStepRunner()
mockApply := mocks.NewMockStepRunner()
mockRun := mocks.NewMockStepRunner()
mockRun := mocks.NewMockCustomStepRunner()
mockApproved := mocks2.NewMockPullApprovedChecker()
mockWorkingDir := mocks.NewMockWorkingDir()
mockLocker := mocks.NewMockProjectLocker()
@@ -287,7 +287,7 @@ func TestDefaultProjectCommandRunner_Apply(t *testing.T) {
When(mockInit.Run(ctx, nil, repoDir)).ThenReturn("init", nil)
When(mockPlan.Run(ctx, nil, repoDir)).ThenReturn("plan", nil)
When(mockApply.Run(ctx, nil, repoDir)).ThenReturn("apply", nil)
When(mockRun.Run(ctx, nil, repoDir)).ThenReturn("run", nil)
When(mockRun.Run(ctx, "", repoDir)).ThenReturn("run", nil)
When(mockApproved.PullIsApproved(ctx.BaseRepo, ctx.Pull)).ThenReturn(true, nil)
res := runner.Apply(ctx)
@@ -305,7 +305,7 @@ func TestDefaultProjectCommandRunner_Apply(t *testing.T) {
case "apply":
mockApply.VerifyWasCalledOnce().Run(ctx, nil, repoDir)
case "run":
mockRun.VerifyWasCalledOnce().Run(ctx, nil, repoDir)
mockRun.VerifyWasCalledOnce().Run(ctx, "", repoDir)
}
}
})

View File

@@ -2,14 +2,11 @@ package runtime
import (
"fmt"
"github.com/hashicorp/go-version"
"github.com/runatlantis/atlantis/server/events/models"
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/hashicorp/go-version"
"github.com/pkg/errors"
"github.com/runatlantis/atlantis/server/events/models"
)
// RunStepRunner runs custom commands.
@@ -17,12 +14,8 @@ type RunStepRunner struct {
DefaultTFVersion *version.Version
}
func (r *RunStepRunner) Run(ctx models.ProjectCommandContext, command []string, path string) (string, error) {
if len(command) < 1 {
return "", errors.New("no commands for run step")
}
cmd := exec.Command("sh", "-c", strings.Join(command, " ")) // #nosec
func (r *RunStepRunner) Run(ctx models.ProjectCommandContext, command string, path string) (string, error) {
cmd := exec.Command("sh", "-c", command) // #nosec
cmd.Dir = path
tfVersion := r.DefaultTFVersion.String()
if ctx.TerraformVersion != nil {
@@ -52,12 +45,11 @@ func (r *RunStepRunner) Run(ctx models.ProjectCommandContext, command []string,
cmd.Env = finalEnvVars
out, err := cmd.CombinedOutput()
commandStr := strings.Join(command, " ")
if err != nil {
err = fmt.Errorf("%s: running %q in %q: \n%s", err, commandStr, path, out)
err = fmt.Errorf("%s: running %q in %q: \n%s", err, command, path, out)
ctx.Log.Debug("error: %s", err)
return string(out), err
}
ctx.Log.Info("successfully ran %q in %q", commandStr, path)
ctx.Log.Info("successfully ran %q in %q", command, path)
return string(out), nil
}

View File

@@ -19,12 +19,24 @@ func TestRunStepRunner_Run(t *testing.T) {
}{
{
Command: "",
ExpErr: "no commands for run step",
ExpOut: "",
},
{
Command: "echo hi",
ExpOut: "hi\n",
},
{
Command: `printf \'your main.tf file does not provide default region.\\ncheck\'`,
ExpOut: `'your`,
},
{
Command: `printf 'your main.tf file does not provide default region.\ncheck'`,
ExpOut: "your main.tf file does not provide default region.\ncheck",
},
{
Command: "echo 'a",
ExpErr: "exit status 2: running \"echo 'a\" in",
},
{
Command: "echo hi >> file && cat file",
ExpOut: "hi\n",
@@ -80,11 +92,7 @@ func TestRunStepRunner_Run(t *testing.T) {
t.Run(c.Command, func(t *testing.T) {
tmpDir, cleanup := TempDir(t)
defer cleanup()
var split []string
if c.Command != "" {
split = strings.Split(c.Command, " ")
}
out, err := r.Run(ctx, split, tmpDir)
out, err := r.Run(ctx, c.Command, tmpDir)
if c.ExpErr != "" {
ErrContains(t, c.ExpErr, err)
return

View File

@@ -3,9 +3,11 @@ package yaml
import (
"encoding/json"
"fmt"
"github.com/flynn-archive/go-shlex"
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/go-ozzo/ozzo-validation"
"github.com/pkg/errors"
@@ -66,6 +68,14 @@ func (p *ParserValidator) ParseRepoCfg(absRepoDir string, globalCfg valid.Global
if err := p.validateProjectNames(validConfig); err != nil {
return valid.RepoCfg{}, err
}
if validConfig.Version == 2 {
// The only difference between v2 and v3 is how we parse custom run
// commands.
if err := p.applyLegacyShellParsing(&validConfig); err != nil {
return validConfig, err
}
}
err = globalCfg.ValidateRepoCfg(validConfig, repoID)
return validConfig, err
}
@@ -161,3 +171,36 @@ func (p *ParserValidator) validateProjectNames(config valid.RepoCfg) error {
return nil
}
// applyLegacyShellParsing changes any custom run commands in cfg to use the old
// parsing method with shlex.Split().
func (p *ParserValidator) applyLegacyShellParsing(cfg *valid.RepoCfg) error {
legacyParseF := func(s *valid.Step) error {
if s.StepName == "run" {
split, err := shlex.Split(s.RunCommand)
if err != nil {
return errors.Wrapf(err, "unable to parse %q", s.RunCommand)
}
s.RunCommand = strings.Join(split, " ")
}
return nil
}
for k := range cfg.Workflows {
w := cfg.Workflows[k]
for i := range w.Plan.Steps {
s := &w.Plan.Steps[i]
if err := legacyParseF(s); err != nil {
return err
}
}
for i := range w.Apply.Steps {
s := &w.Apply.Steps[i]
if err := legacyParseF(s); err != nil {
return err
}
}
cfg.Workflows[k] = w
}
return nil
}

View File

@@ -1,6 +1,7 @@
package yaml_test
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
@@ -110,7 +111,7 @@ func TestParseRepoCfg(t *testing.T) {
projects:
- dir: "."
`,
expErr: "version: is required. If you've just upgraded Atlantis you need to rewrite your atlantis.yaml for version 2. See www.runatlantis.io/docs/upgrading-atlantis-yaml-to-version-2.html.",
expErr: "version: is required. If you've just upgraded Atlantis you need to rewrite your atlantis.yaml for version 3. See www.runatlantis.io/docs/upgrading-atlantis-yaml.html.",
},
{
description: "unsupported version",
@@ -119,7 +120,7 @@ version: 0
projects:
- dir: "."
`,
expErr: "version: must equal 2.",
expErr: "version: only versions 2 and 3 are supported.",
},
{
description: "empty version",
@@ -128,17 +129,45 @@ version: ~
projects:
- dir: "."
`,
expErr: "version: must equal 2.",
expErr: "version: only versions 2 and 3 are supported.",
},
{
description: "version 2",
input: `
version: 2
workflows:
custom:
plan:
steps:
- run: old 'shell parsing'
`,
exp: valid.RepoCfg{
Version: 2,
Workflows: map[string]valid.Workflow{
"custom": {
Name: "custom",
Apply: valid.DefaultApplyStage,
Plan: valid.Stage{
Steps: []valid.Step{
{
StepName: "run",
RunCommand: "old shell parsing",
},
},
},
},
},
},
},
// Projects key.
{
description: "empty projects list",
input: `
version: 2
version: 3
projects:`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: nil,
Workflows: map[string]valid.Workflow{},
},
@@ -146,7 +175,7 @@ projects:`,
{
description: "project dir not set",
input: `
version: 2
version: 3
projects:
- `,
expErr: "projects: (0: (dir: cannot be blank.).).",
@@ -154,11 +183,11 @@ projects:
{
description: "project dir set",
input: `
version: 2
version: 3
projects:
- dir: .`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -178,14 +207,14 @@ projects:
{
description: "autoplan should be enabled by default",
input: `
version: 2
version: 3
projects:
- dir: "."
autoplan:
when_modified: ["**/*.tf*"]
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -202,12 +231,12 @@ projects:
{
description: "if workflows not defined there are none",
input: `
version: 2
version: 3
projects:
- dir: "."
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -224,13 +253,13 @@ projects:
{
description: "if workflows key set but with no workflows there are none",
input: `
version: 2
version: 3
projects:
- dir: "."
workflows: ~
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -247,7 +276,7 @@ workflows: ~
{
description: "if a plan or apply explicitly defines an empty steps key then it gets the defaults",
input: `
version: 2
version: 3
projects:
- dir: "."
workflows:
@@ -258,7 +287,7 @@ workflows:
steps:
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -281,7 +310,7 @@ workflows:
{
description: "project fields set except autoplan",
input: `
version: 2
version: 3
projects:
- dir: .
workspace: myworkspace
@@ -291,7 +320,7 @@ projects:
workflows:
myworkflow: ~`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -317,7 +346,7 @@ workflows:
{
description: "project field with autoplan",
input: `
version: 2
version: 3
projects:
- dir: .
workspace: myworkspace
@@ -329,7 +358,7 @@ projects:
workflows:
myworkflow: ~`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -355,7 +384,7 @@ workflows:
{
description: "project field with mergeable apply requirement",
input: `
version: 2
version: 3
projects:
- dir: .
workspace: myworkspace
@@ -367,7 +396,7 @@ projects:
workflows:
myworkflow: ~`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -393,7 +422,7 @@ workflows:
{
description: "project field with mergeable and approved apply requirements",
input: `
version: 2
version: 3
projects:
- dir: .
workspace: myworkspace
@@ -405,7 +434,7 @@ projects:
workflows:
myworkflow: ~`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -431,7 +460,7 @@ workflows:
{
description: "project dir with ..",
input: `
version: 2
version: 3
projects:
- dir: ..`,
expErr: "projects: (0: (dir: cannot contain '..'.).).",
@@ -441,7 +470,7 @@ projects:
{
description: "project with no config",
input: `
version: 2
version: 3
projects:
-`,
expErr: "projects: (0: (dir: cannot be blank.).).",
@@ -449,7 +478,7 @@ projects:
{
description: "project with no config at index 1",
input: `
version: 2
version: 3
projects:
- dir: "."
-`,
@@ -458,7 +487,7 @@ projects:
{
description: "project with unknown key",
input: `
version: 2
version: 3
projects:
- unknown: value`,
expErr: "yaml: unmarshal errors:\n line 4: field unknown not found in struct raw.Project",
@@ -466,7 +495,7 @@ projects:
{
description: "referencing workflow that doesn't exist",
input: `
version: 2
version: 3
projects:
- dir: .
workflow: undefined`,
@@ -475,7 +504,7 @@ projects:
{
description: "two projects with same dir/workspace without names",
input: `
version: 2
version: 3
projects:
- dir: .
workspace: workspace
@@ -486,7 +515,7 @@ projects:
{
description: "two projects with same dir/workspace only one with name",
input: `
version: 2
version: 3
projects:
- name: myname
dir: .
@@ -498,7 +527,7 @@ projects:
{
description: "two projects with same dir/workspace both with same name",
input: `
version: 2
version: 3
projects:
- name: myname
dir: .
@@ -511,7 +540,7 @@ projects:
{
description: "two projects with same dir/workspace with different names",
input: `
version: 2
version: 3
projects:
- name: myname
dir: .
@@ -520,7 +549,7 @@ projects:
dir: .
workspace: workspace`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Name: String("myname"),
@@ -547,7 +576,7 @@ projects:
{
description: "if steps are set then we parse them properly",
input: `
version: 2
version: 3
projects:
- dir: "."
workflows:
@@ -562,7 +591,7 @@ workflows:
- apply
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -603,7 +632,7 @@ workflows:
{
description: "we parse extra_args for the steps",
input: `
version: 2
version: 3
projects:
- dir: "."
workflows:
@@ -624,7 +653,7 @@ workflows:
extra_args: ["a", "b"]
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -669,7 +698,7 @@ workflows:
{
description: "custom steps are parsed",
input: `
version: 2
version: 3
projects:
- dir: "."
workflows:
@@ -682,7 +711,7 @@ workflows:
- run: echo apply "arg 2"
`,
exp: valid.RepoCfg{
Version: 2,
Version: 3,
Projects: []valid.Project{
{
Dir: ".",
@@ -700,7 +729,7 @@ workflows:
Steps: []valid.Step{
{
StepName: "run",
RunCommand: []string{"echo", "plan hi"},
RunCommand: "echo \"plan hi\"",
},
},
},
@@ -708,7 +737,7 @@ workflows:
Steps: []valid.Step{
{
StepName: "run",
RunCommand: []string{"echo", "apply", "arg 2"},
RunCommand: "echo apply \"arg 2\"",
},
},
},
@@ -745,7 +774,7 @@ func TestParseRepoCfg_GlobalValidation(t *testing.T) {
defer cleanup()
repoCfg := `
version: 2
version: 3
projects:
- dir: .
workflow: custom
@@ -773,7 +802,7 @@ func TestParseGlobalCfg(t *testing.T) {
Steps: []valid.Step{
{
StepName: "run",
RunCommand: []string{"custom", "command"},
RunCommand: "custom command",
},
{
StepName: "init",
@@ -788,7 +817,7 @@ func TestParseGlobalCfg(t *testing.T) {
Steps: []valid.Step{
{
StepName: "run",
RunCommand: []string{"custom", "command"},
RunCommand: "custom command",
},
{
StepName: "apply",
@@ -1005,7 +1034,7 @@ func TestParserValidator_ParseGlobalCfgJSON(t *testing.T) {
},
{
StepName: "run",
RunCommand: []string{"custom", "plan"},
RunCommand: "custom plan",
},
},
},
@@ -1013,7 +1042,7 @@ func TestParserValidator_ParseGlobalCfgJSON(t *testing.T) {
Steps: []valid.Step{
{
StepName: "run",
RunCommand: []string{"my", "custom", "command"},
RunCommand: "my custom command",
},
},
},
@@ -1104,6 +1133,68 @@ func TestParserValidator_ParseGlobalCfgJSON(t *testing.T) {
}
}
// Test legacy shell parsing vs v3 parsing.
func TestParseRepoCfg_V2ShellParsing(t *testing.T) {
cases := []struct {
in string
expV2 string
expV2Err string
}{
{
in: "echo a b",
expV2: "echo a b",
},
{
in: "echo 'a b'",
expV2: "echo a b",
},
{
in: "echo 'a b",
expV2Err: "unable to parse \"echo 'a b\": EOF found when expecting closing quote.",
},
{
in: `mkdir a/b/c || printf \'your main.tf file does not provide default region.\\ncheck\'`,
expV2: `mkdir a/b/c || printf 'your main.tf file does not provide default region.\ncheck'`,
},
}
for _, c := range cases {
t.Run(c.in, func(t *testing.T) {
v2Dir, cleanup2 := TempDir(t)
defer cleanup2()
v3Dir, cleanup3 := TempDir(t)
defer cleanup3()
v2Path := filepath.Join(v2Dir, "atlantis.yaml")
v3Path := filepath.Join(v3Dir, "atlantis.yaml")
cfg := fmt.Sprintf(`workflows:
custom:
plan:
steps:
- run: %s
apply:
steps:
- run: %s`, c.in, c.in)
Ok(t, ioutil.WriteFile(v2Path, []byte("version: 2\n"+cfg), 0600))
Ok(t, ioutil.WriteFile(v3Path, []byte("version: 3\n"+cfg), 0600))
p := &yaml.ParserValidator{}
v2Cfg, err := p.ParseRepoCfg(v2Dir, valid.NewGlobalCfg(true, false, false), "")
if c.expV2Err != "" {
ErrEquals(t, c.expV2Err, err)
} else {
Ok(t, err)
Equals(t, c.expV2, v2Cfg.Workflows["custom"].Plan.Steps[0].RunCommand)
Equals(t, c.expV2, v2Cfg.Workflows["custom"].Apply.Steps[0].RunCommand)
}
v3Cfg, err := p.ParseRepoCfg(v3Dir, valid.NewGlobalCfg(true, false, false), "")
Ok(t, err)
Equals(t, c.in, v3Cfg.Workflows["custom"].Plan.Steps[0].RunCommand)
Equals(t, c.in, v3Cfg.Workflows["custom"].Apply.Steps[0].RunCommand)
})
}
}
// String is a helper routine that allocates a new string value
// to store v and returns a pointer to it.
func String(v string) *string { return &v }

View File

@@ -22,10 +22,10 @@ func (r RepoCfg) Validate() error {
equals2 := func(value interface{}) error {
asIntPtr := value.(*int)
if asIntPtr == nil {
return errors.New("is required. If you've just upgraded Atlantis you need to rewrite your atlantis.yaml for version 2. See www.runatlantis.io/docs/upgrading-atlantis-yaml-to-version-2.html")
return errors.New("is required. If you've just upgraded Atlantis you need to rewrite your atlantis.yaml for version 3. See www.runatlantis.io/docs/upgrading-atlantis-yaml.html")
}
if *asIntPtr != 2 {
return errors.New("must equal 2")
if *asIntPtr != 2 && *asIntPtr != 3 {
return errors.New("only versions 2 and 3 are supported")
}
return nil
}

View File

@@ -46,7 +46,7 @@ func TestConfig_UnmarshalYAML(t *testing.T) {
expErr: "yaml: unmarshal errors:\n line 1: field invalid not found in struct raw.RepoCfg",
},
{
description: "version set",
description: "version set to 2",
input: "version: 2",
exp: raw.RepoCfg{
Version: Int(2),
@@ -54,6 +54,15 @@ func TestConfig_UnmarshalYAML(t *testing.T) {
Workflows: nil,
},
},
{
description: "version set to 3",
input: "version: 3",
exp: raw.RepoCfg{
Version: Int(3),
Projects: nil,
Workflows: nil,
},
},
{
description: "projects key without value",
input: "projects:",
@@ -94,7 +103,7 @@ func TestConfig_UnmarshalYAML(t *testing.T) {
},
{
description: "automerge not a boolean",
input: "version: 2\nautomerge: notabool",
input: "version: 3\nautomerge: notabool",
exp: raw.RepoCfg{
Version: nil,
Projects: nil,
@@ -105,7 +114,7 @@ func TestConfig_UnmarshalYAML(t *testing.T) {
{
description: "should use values if set",
input: `
version: 2
version: 3
automerge: true
projects:
- dir: mydir
@@ -123,7 +132,7 @@ workflows:
apply:
steps: []`,
exp: raw.RepoCfg{
Version: Int(2),
Version: Int(3),
Automerge: Bool(true),
Projects: []raw.Project{
{
@@ -176,14 +185,14 @@ func TestConfig_Validate(t *testing.T) {
input: raw.RepoCfg{
Version: nil,
},
expErr: "version: is required. If you've just upgraded Atlantis you need to rewrite your atlantis.yaml for version 2. See www.runatlantis.io/docs/upgrading-atlantis-yaml-to-version-2.html.",
expErr: "version: is required. If you've just upgraded Atlantis you need to rewrite your atlantis.yaml for version 3. See www.runatlantis.io/docs/upgrading-atlantis-yaml.html.",
},
{
description: "version not 1",
description: "version not 2 or 3",
input: raw.RepoCfg{
Version: Int(1),
},
expErr: "version: must equal 2.",
expErr: "version: only versions 2 and 3 are supported.",
},
}
validation.ErrorTag = "yaml"

View File

@@ -7,7 +7,6 @@ import (
"sort"
"strings"
"github.com/flynn-archive/go-shlex"
"github.com/go-ozzo/ozzo-validation"
"github.com/runatlantis/atlantis/server/events/yaml/valid"
)
@@ -108,14 +107,10 @@ func (s Step) Validate() error {
return fmt.Errorf("step element can only contain a single key, found %d: %s",
len(keys), strings.Join(keys, ","))
}
for stepName, args := range elem {
for stepName := range elem {
if stepName != RunStepName {
return fmt.Errorf("%q is not a valid step type", stepName)
}
_, err := shlex.Split(args)
if err != nil {
return fmt.Errorf("unable to parse as shell command: %s", err)
}
}
return nil
}
@@ -157,12 +152,9 @@ func (s Step) ToValid() valid.Step {
// After validation we assume there's only one key and it's a valid
// step name so we just use the first one.
for _, v := range s.StringVal {
// We ignore the error here because it should have been checked in
// Validate().
split, _ := shlex.Split(v)
return valid.Step{
StepName: RunStepName,
RunCommand: split,
RunCommand: v,
}
}
}

View File

@@ -268,13 +268,14 @@ func TestStep_Validate(t *testing.T) {
expErr: "built-in steps only support a single extra_args key, found \"invalid\" in step init",
},
{
// For atlantis.yaml v2, this wouldn't parse, but now there should
// be no error.
description: "unparseable shell command",
input: raw.Step{
StringVal: map[string]string{
"run": "my 'c",
},
},
expErr: "unable to parse as shell command: EOF found when expecting closing quote.",
},
}
for _, c := range cases {
@@ -373,7 +374,7 @@ func TestStep_ToValid(t *testing.T) {
},
exp: valid.Step{
StepName: "run",
RunCommand: []string{"my", "run command"},
RunCommand: "my 'run command'",
},
},
}

View File

@@ -45,6 +45,7 @@ type MergedProjectCfg struct {
Name string
AutoplanEnabled bool
TerraformVersion *version.Version
RepoCfgVersion int
}
// DefaultApplyStage is the Atlantis default apply stage.
@@ -178,6 +179,7 @@ func (g GlobalCfg) MergeProjectCfg(log logging.SimpleLogging, repoID string, pro
Name: proj.GetName(),
AutoplanEnabled: proj.Autoplan.Enabled,
TerraformVersion: proj.TerraformVersion,
RepoCfgVersion: rCfg.Version,
}
}

View File

@@ -6,8 +6,7 @@ import "github.com/hashicorp/go-version"
// RepoCfg is the atlantis.yaml config after it's been parsed and validated.
type RepoCfg struct {
// Version is the version of the atlantis YAML file. Will always be equal
// to 2.
// Version is the version of the atlantis YAML file.
Version int
Projects []Project
Workflows map[string]Workflow
@@ -75,7 +74,7 @@ type Stage struct {
type Step struct {
StepName string
ExtraArgs []string
RunCommand []string
RunCommand string
}
type Workflow struct {

View File

@@ -1,4 +1,4 @@
version: 2
version: 3
automerge: true
projects:
- dir: dir1

View File

@@ -1,4 +1,4 @@
version: 2
version: 3
projects:
- dir: staging
autoplan:

View File

@@ -1,4 +1,4 @@
version: 2
version: 3
projects:
- dir: .
workspace: default

View File

@@ -1,4 +1,4 @@
version: 2
version: 3
projects:
- dir: .
name: default

View File

@@ -1,4 +1,4 @@
version: 2
version: 3
projects:
- dir: .
name: default