public/atlantis
1
0
Fork 0
mirror of https://git.vectorsigma.ru/public/atlantis.git synced 2026-07-29 01:28:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Add option allowing usage of custom policy check tools (#3765)

Browse Source 
* Adding new flag everywhere relevant, implementing policy result workaround

* Fixing unit test str matching, adding custom policy conditional to step_runner

* Adding documentation steps for custom policy tools

* Refactoring ConftestOutput attribute to PolicyOutput
This commit is contained in:
Brett Galkowski
2023-10-06 07:57:29 -07:00
committed by GitHub
parent cf2b7911f4
commit 22060fe5d2
20 changed files with 200 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
runatlantis.io/docs/custom-policy-checks.md Normal file
View File

@@ -0,0 +1,45 @@
# Custom Policy Checks
If you want to run custom policy tools or scripts instead of the built-in Conftest integration, you can do so by setting the `custom_policy_check` option and running it in a custom workflow. Note: custom policy tool output is simply parsed for "fail" substrings to determine if the policy set passed.
This option can be configured either at the server-level in a [repos.yaml config file](server-configuration.md) or at the repo-level in an [atlantis.yaml file.](repo-level-atlantis-yaml.md).
## Server-side config example
Set the `policy_check` and `custom_policy_check` options to true, and run the custom tool in the policy check steps as seen below. No
```yaml
repos:
- id: /.*/
branch: /^main$/
apply_requirements: [mergeable, undiverged, approved]
policy_check: true
custom_policy_check: true
workflow: custom
workflows:
custom:
policy_check:
steps:
- show
- run: cnspec scan terraform plan $SHOWFILE --policy-bundle example-cnspec-policies.mql.yaml
policies:
owners:
users:
- example_ghuser
policy_sets:
- name: example-set
path: example-cnspec-policies.mql.yaml
source: local
```
## Repo-level atlantis.yaml example
First, you will need to ensure `custom_policy_check` is within the `allowed_overrides` field of the server-side config. Next, just set the custom option to true on the specific project you want as shown in the example `atlantis.yaml` below:
```yaml
version: 3
projects:
- name: example
dir: ./example
custom_policy_check: true
autoplan:
when_modified: ["*.tf"]
```

2
runatlantis.io/docs/policy-checking.md
View File

@@ -187,7 +187,7 @@ When the policy check workflow runs, a file is created in the working directory
[
{
"PolicySetName": "policy1",
"ConftestOutput": "",
"PolicyOutput": "",
"Passed": false,
"ReqApprovals": 1,
"CurApprovals": 0

6
runatlantis.io/docs/repo-level-atlantis-yaml.md
View File

@@ -59,6 +59,7 @@ projects:
terraform_version: v0.11.0
delete_source_branch_on_merge: true
repo_locking: true
custom_policy_check: false
autoplan:
when_modified: ["*.tf", "../modules/**/*.tf", ".terraform.lock.hcl"]
enabled: true
@@ -311,6 +312,7 @@ workspace: myworkspace
execution_order_group: 0
delete_source_branch_on_merge: false
repo_locking: true
custom_policy_check: false
autoplan:
terraform_version: 0.11.0
plan_requirements: ["approved"]
@@ -327,7 +329,9 @@ workflow: myworkflow
| workspace | string | `"default"` | no | The [Terraform workspace](https://developer.hashicorp.com/terraform/language/state/workspaces) for this project. Atlantis will switch to this workplace when planning/applying and will create it if it doesn't exist. |
| execution_order_group | int | `0` | no | Index of execution order group. Projects will be sort by this field before planning/applying. |
| delete_source_branch_on_merge | bool | `false` | no | Automatically deletes the source branch on merge. |
| repo_locking | bool | `true` | no | Get a repository lock in this project when plan. |
| repo_locking | bool | `true` | no | Get a repository lock in this project when plan.
| custom_policy_check | bool | `false` | no | Enable using policy check tools other than Conftest |
| autoplan | [Autoplan](#autoplan) | none | no | A custom autoplan configuration. If not specified, will use the autoplan config. See [Autoplanning](autoplanning.html). |
| terraform_version | string | none | no | A specific Terraform version to use when running commands for this project. Must be [Semver compatible](https://semver.org/), ex. `v0.11.0`, `0.12.0-beta1`. |
| plan_requirements<br />*(restricted)* | array[string] | none | no | Requirements that must be satisfied before `atlantis plan` can be run. Currently the only supported requirements are `approved`, `mergeable`, and `undiverged`. See [Command Requirements](command-requirements.html) for more details. |

12
runatlantis.io/docs/server-side-repo-config.md
View File

@@ -54,7 +54,7 @@ repos:
# allowed_overrides specifies which keys can be overridden by this repo in
# its atlantis.yaml file.
allowed_overrides: [apply_requirements, workflow, delete_source_branch_on_merge, repo_locking]
allowed_overrides: [apply_requirements, workflow, delete_source_branch_on_merge, repo_locking, custom_policy_check]
# allowed_workflows specifies which workflows the repos that match
# are allowed to select.
@@ -73,6 +73,10 @@ repos:
# If true (default), atlantis try to get a lock.
repo_locking: true
# custom_policy_check defines whether policy checking tools besides Conftest are enabled in checks
# If false (default), only Conftest JSON output is allowed
custom_policy_check: false
# pre_workflow_hooks defines arbitrary list of scripts to execute before workflow execution.
pre_workflow_hooks:
- run: my-pre-workflow-hook-command arg1
@@ -340,6 +344,9 @@ unless you've created your own server-side workflow with that key (overriding it
See [Custom Workflows](custom-workflows.html) for more details on writing
custom workflows.
### Allow Using Custom Policy Tools
Conftest is the standard policy check application integrated with Atlantis, but custom tools can still be run in custom workflows when the `custom_policy_check` option is set. See the [Custom Policy Checks page](custom-policy-checks.md) for detailed examples.
### Allow Repos To Define Their Own Workflows
If you want repos to be able to define their own workflows you need to
allow them to override the `workflow` key and set `allow_custom_workflows` to `true`.
@@ -482,12 +489,13 @@ If you set a workflow with the key `default`, it will override this.
| plan_requirements | []string | none | no | Requirements that must be satisfied before `atlantis plan` can be run. Currently the only supported requirements are `approved`, `mergeable`, and `undiverged`. See [Command Requirements](command-requirements.html) for more details. | |
| apply_requirements | []string | none | no | Requirements that must be satisfied before `atlantis apply` can be run. Currently the only supported requirements are `approved`, `mergeable`, and `undiverged`. See [Command Requirements](command-requirements.html) for more details. |
| import_requirements | []string | none | no | Requirements that must be satisfied before `atlantis import` can be run. Currently the only supported requirements are `approved`, `mergeable`, and `undiverged`. See [Command Requirements](command-requirements.html) for more details. |
| allowed_overrides | []string | none | no | A list of restricted keys that `atlantis.yaml` files can override. The only supported keys are `apply_requirements`, `workflow`, `delete_source_branch_on_merge` and `repo_locking` |
| allowed_overrides | []string | none | no | A list of restricted keys that `atlantis.yaml` files can override. The only supported keys are `apply_requirements`, `workflow`, `delete_source_branch_on_merge`,`repo_locking`, and `custom_policy_check` |
| allowed_workflows | []string | none | no | A list of workflows that `atlantis.yaml` files can select from. |
| allow_custom_workflows | bool | false | no | Whether or not to allow [Custom Workflows](custom-workflows.html). |
| delete_source_branch_on_merge | bool | false | no | Whether or not to delete the source branch on merge. |
| repo_locking | bool | false | no | Whether or not to get a lock. |
| policy_check | bool | false | no | Whether or not to run policy checks on this repository. |
| custom_policy_check | bool | false | no | Whether or not to enable custom policy check tools outside of Conftest on this repository. |
:::tip Notes

3
server/core/config/parser_validator_test.go
View File

@@ -1309,7 +1309,7 @@ func TestParseGlobalCfg(t *testing.T) {
input: `repos:
- id: /.*/
allowed_overrides: [invalid]`,
expErr: "repos: (0: (allowed_overrides: \"invalid\" is not a valid override, only \"plan_requirements\", \"apply_requirements\", \"import_requirements\", \"workflow\", \"delete_source_branch_on_merge\", \"repo_locking\" and \"policy_check\" are supported.).).",
expErr: "repos: (0: (allowed_overrides: \"invalid\" is not a valid override, only \"plan_requirements\", \"apply_requirements\", \"import_requirements\", \"workflow\", \"delete_source_branch_on_merge\", \"repo_locking\", \"policy_check\", and \"custom_policy_check\" are supported.).).",
},
"invalid plan_requirement": {
input: `repos:
@@ -1573,6 +1573,7 @@ workflows:
DeleteSourceBranchOnMerge: Bool(false),
RepoLocking: Bool(true),
PolicyCheck: Bool(false),
CustomPolicyCheck: Bool(false),
},
},
Workflows: map[string]valid.Workflow{

6
server/core/config/raw/global_cfg.go
View File

@@ -35,6 +35,7 @@ type Repo struct {
DeleteSourceBranchOnMerge *bool `yaml:"delete_source_branch_on_merge,omitempty" json:"delete_source_branch_on_merge,omitempty"`
RepoLocking *bool `yaml:"repo_locking,omitempty" json:"repo_locking,omitempty"`
PolicyCheck *bool `yaml:"policy_check,omitempty" json:"policy_check,omitempty"`
CustomPolicyCheck *bool `yaml:"custom_policy_check,omitempty" json:"custom_policy_check,omitempty"`
}
func (g GlobalCfg) Validate() error {
@@ -192,8 +193,8 @@ func (r Repo) Validate() error {
overridesValid := func(value interface{}) error {
overrides := value.([]string)
for _, o := range overrides {
if o != valid.PlanRequirementsKey && o != valid.ApplyRequirementsKey && o != valid.ImportRequirementsKey && o != valid.WorkflowKey && o != valid.DeleteSourceBranchOnMergeKey && o != valid.RepoLockingKey && o != valid.PolicyCheckKey {
return fmt.Errorf("%q is not a valid override, only %q, %q, %q, %q, %q, %q and %q are supported", o, valid.PlanRequirementsKey, valid.ApplyRequirementsKey, valid.ImportRequirementsKey, valid.WorkflowKey, valid.DeleteSourceBranchOnMergeKey, valid.RepoLockingKey, valid.PolicyCheckKey)
if o != valid.PlanRequirementsKey && o != valid.ApplyRequirementsKey && o != valid.ImportRequirementsKey && o != valid.WorkflowKey && o != valid.DeleteSourceBranchOnMergeKey && o != valid.RepoLockingKey && o != valid.PolicyCheckKey && o != valid.CustomPolicyCheckKey {
return fmt.Errorf("%q is not a valid override, only %q, %q, %q, %q, %q, %q, %q, and %q are supported", o, valid.PlanRequirementsKey, valid.ApplyRequirementsKey, valid.ImportRequirementsKey, valid.WorkflowKey, valid.DeleteSourceBranchOnMergeKey, valid.RepoLockingKey, valid.PolicyCheckKey, valid.CustomPolicyCheckKey)
}
}
return nil
@@ -331,5 +332,6 @@ OuterGlobalImportReqs:
DeleteSourceBranchOnMerge: r.DeleteSourceBranchOnMerge,
RepoLocking: r.RepoLocking,
PolicyCheck: r.PolicyCheck,
CustomPolicyCheck: r.CustomPolicyCheck,
}
}

5
server/core/config/raw/project.go
View File

@@ -36,6 +36,7 @@ type Project struct {
RepoLocking *bool `yaml:"repo_locking,omitempty"`
ExecutionOrderGroup *int `yaml:"execution_order_group,omitempty"`
PolicyCheck *bool `yaml:"policy_check,omitempty"`
CustomPolicyCheck *bool `yaml:"custom_policy_check,omitempty"`
}
func (p Project) Validate() error {
@@ -146,6 +147,10 @@ func (p Project) ToValid() valid.Project {
v.PolicyCheck = p.PolicyCheck
}
if p.CustomPolicyCheck != nil {
v.CustomPolicyCheck = p.CustomPolicyCheck
}
return v
}

28
server/core/config/valid/global_cfg.go
View File

@@ -24,6 +24,7 @@ const DefaultWorkflowName = "default"
const DeleteSourceBranchOnMergeKey = "delete_source_branch_on_merge"
const RepoLockingKey = "repo_locking"
const PolicyCheckKey = "policy_check"
const CustomPolicyCheckKey = "custom_policy_check"
// DefaultAtlantisFile is the default name of the config file for each repo.
const DefaultAtlantisFile = "atlantis.yaml"
@@ -79,6 +80,7 @@ type Repo struct {
DeleteSourceBranchOnMerge *bool
RepoLocking *bool
PolicyCheck *bool
CustomPolicyCheck *bool
}
type MergedProjectCfg struct {
@@ -100,6 +102,7 @@ type MergedProjectCfg struct {
ExecutionOrderGroup int
RepoLocking bool
PolicyCheck bool
CustomPolicyCheck bool
}
// WorkflowHook is a map of custom run commands to run before or after workflows.
@@ -239,6 +242,7 @@ func NewGlobalCfgFromArgs(args GlobalCfgArgs) GlobalCfg {
allowCustomWorkflows := false
deleteSourceBranchOnMerge := false
repoLockingKey := true
customPolicyCheck := false
if args.AllowRepoCfg {
allowedOverrides = []string{PlanRequirementsKey, ApplyRequirementsKey, ImportRequirementsKey, WorkflowKey, DeleteSourceBranchOnMergeKey, RepoLockingKey, PolicyCheckKey}
allowCustomWorkflows = true
@@ -262,6 +266,7 @@ func NewGlobalCfgFromArgs(args GlobalCfgArgs) GlobalCfg {
DeleteSourceBranchOnMerge: &deleteSourceBranchOnMerge,
RepoLocking: &repoLockingKey,
PolicyCheck: &policyCheck,
CustomPolicyCheck: &customPolicyCheck,
},
},
Workflows: map[string]Workflow{
@@ -298,7 +303,7 @@ func (r Repo) IDString() string {
// final config. It assumes that all configs have been validated.
func (g GlobalCfg) MergeProjectCfg(log logging.SimpleLogging, repoID string, proj Project, rCfg RepoCfg) MergedProjectCfg {
log.Debug("MergeProjectCfg started")
planReqs, applyReqs, importReqs, workflow, allowedOverrides, allowCustomWorkflows, deleteSourceBranchOnMerge, repoLocking, policyCheck := g.getMatchingCfg(log, repoID)
planReqs, applyReqs, importReqs, workflow, allowedOverrides, allowCustomWorkflows, deleteSourceBranchOnMerge, repoLocking, policyCheck, customPolicyCheck := g.getMatchingCfg(log, repoID)
// If repos are allowed to override certain keys then override them.
for _, key := range allowedOverrides {
@@ -364,6 +369,11 @@ func (g GlobalCfg) MergeProjectCfg(log logging.SimpleLogging, repoID string, pro
log.Debug("overriding server-defined %s with repo settings: [%t]", PolicyCheckKey, *proj.PolicyCheck)
policyCheck = *proj.PolicyCheck
}
case CustomPolicyCheckKey:
if proj.CustomPolicyCheck != nil {
log.Debug("overriding server-defined %s with repo settings: [%t]", CustomPolicyCheckKey, *proj.CustomPolicyCheck)
customPolicyCheck = *proj.CustomPolicyCheck
}
}
log.Debug("MergeProjectCfg completed")
}
@@ -388,6 +398,7 @@ func (g GlobalCfg) MergeProjectCfg(log logging.SimpleLogging, repoID string, pro
ExecutionOrderGroup: proj.ExecutionOrderGroup,
RepoLocking: repoLocking,
PolicyCheck: policyCheck,
CustomPolicyCheck: customPolicyCheck,
}
}
@@ -395,7 +406,7 @@ func (g GlobalCfg) MergeProjectCfg(log logging.SimpleLogging, repoID string, pro
// repo with id repoID. It is used when there is no repo config.
func (g GlobalCfg) DefaultProjCfg(log logging.SimpleLogging, repoID string, repoRelDir string, workspace string) MergedProjectCfg {
log.Debug("building config based on server-side config")
planReqs, applyReqs, importReqs, workflow, _, _, deleteSourceBranchOnMerge, repoLocking, policyCheck := g.getMatchingCfg(log, repoID)
planReqs, applyReqs, importReqs, workflow, _, _, deleteSourceBranchOnMerge, repoLocking, policyCheck, customPolicyCheck := g.getMatchingCfg(log, repoID)
return MergedProjectCfg{
PlanRequirements: planReqs,
ApplyRequirements: applyReqs,
@@ -410,6 +421,7 @@ func (g GlobalCfg) DefaultProjCfg(log logging.SimpleLogging, repoID string, repo
DeleteSourceBranchOnMerge: deleteSourceBranchOnMerge,
RepoLocking: repoLocking,
PolicyCheck: policyCheck,
CustomPolicyCheck: customPolicyCheck,
}
}
@@ -454,6 +466,9 @@ func (g GlobalCfg) ValidateRepoCfg(rCfg RepoCfg, repoID string) error {
if p.RepoLocking != nil && !utils.SlicesContains(allowedOverrides, RepoLockingKey) {
return fmt.Errorf("repo config not allowed to set '%s' key: server-side config needs '%s: [%s]'", RepoLockingKey, AllowedOverridesKey, RepoLockingKey)
}
if p.CustomPolicyCheck != nil && !utils.SlicesContains(allowedOverrides, CustomPolicyCheckKey) {
return fmt.Errorf("repo config not allowed to set '%s' key: server-side config needs '%s: [%s]'", CustomPolicyCheckKey, AllowedOverridesKey, CustomPolicyCheckKey)
}
}
// Check custom workflows.
@@ -512,7 +527,7 @@ func (g GlobalCfg) ValidateRepoCfg(rCfg RepoCfg, repoID string) error {
}
// getMatchingCfg returns the key settings for repoID.
func (g GlobalCfg) getMatchingCfg(log logging.SimpleLogging, repoID string) (planReqs []string, applyReqs []string, importReqs []string, workflow Workflow, allowedOverrides []string, allowCustomWorkflows bool, deleteSourceBranchOnMerge bool, repoLocking bool, policyCheck bool) {
func (g GlobalCfg) getMatchingCfg(log logging.SimpleLogging, repoID string) (planReqs []string, applyReqs []string, importReqs []string, workflow Workflow, allowedOverrides []string, allowCustomWorkflows bool, deleteSourceBranchOnMerge bool, repoLocking bool, policyCheck bool, customPolicyCheck bool) {
toLog := make(map[string]string)
traceF := func(repoIdx int, repoID string, key string, val interface{}) string {
from := "default server config"
@@ -534,7 +549,7 @@ func (g GlobalCfg) getMatchingCfg(log logging.SimpleLogging, repoID string) (pla
return fmt.Sprintf("setting %s: %s from %s", key, valStr, from)
}
for _, key := range []string{PlanRequirementsKey, ApplyRequirementsKey, ImportRequirementsKey, WorkflowKey, AllowedOverridesKey, AllowCustomWorkflowsKey, DeleteSourceBranchOnMergeKey, RepoLockingKey, PolicyCheckKey} {
for _, key := range []string{PlanRequirementsKey, ApplyRequirementsKey, ImportRequirementsKey, WorkflowKey, AllowedOverridesKey, AllowCustomWorkflowsKey, DeleteSourceBranchOnMergeKey, RepoLockingKey, PolicyCheckKey, CustomPolicyCheckKey} {
for i, repo := range g.Repos {
if repo.IDMatches(repoID) {
switch key {
@@ -583,6 +598,11 @@ func (g GlobalCfg) getMatchingCfg(log logging.SimpleLogging, repoID string) (pla
toLog[PolicyCheckKey] = traceF(i, repo.IDString(), PolicyCheckKey, *repo.PolicyCheck)
policyCheck = *repo.PolicyCheck
}
case CustomPolicyCheckKey:
if repo.CustomPolicyCheck != nil {
toLog[CustomPolicyCheckKey] = traceF(i, repo.IDString(), CustomPolicyCheckKey, *repo.CustomPolicyCheck)
customPolicyCheck = *repo.CustomPolicyCheck
}
}
}
}

49
server/core/config/valid/global_cfg_test.go
View File

@@ -81,6 +81,7 @@ func TestNewGlobalCfg(t *testing.T) {
DeleteSourceBranchOnMerge: Bool(false),
RepoLocking: Bool(true),
PolicyCheck: Bool(false),
CustomPolicyCheck: Bool(false),
},
},
Workflows: map[string]valid.Workflow{
@@ -707,11 +708,12 @@ policies:
},
},
},
RepoRelDir: ".",
Workspace: "default",
Name: "",
AutoplanEnabled: false,
RepoLocking: true,
RepoRelDir: ".",
Workspace: "default",
Name: "",
AutoplanEnabled: false,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"policies set correct version if specified": {
@@ -755,11 +757,12 @@ policies:
},
},
},
RepoRelDir: ".",
Workspace: "default",
Name: "",
AutoplanEnabled: false,
RepoLocking: true,
RepoRelDir: ".",
Workspace: "default",
Name: "",
AutoplanEnabled: false,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
}
@@ -848,12 +851,13 @@ workflows:
Import: valid.DefaultImportStage,
StateRm: valid.DefaultStateRmStage,
},
RepoRelDir: ".",
Workspace: "default",
Name: "",
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: true,
RepoRelDir: ".",
Workspace: "default",
Name: "",
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"repo-side plan reqs win out if allowed": {
@@ -883,6 +887,7 @@ repos:
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"repo-side apply reqs win out if allowed": {
@@ -912,6 +917,7 @@ repos:
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"repo-side import reqs win out if allowed": {
@@ -941,6 +947,7 @@ repos:
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"repo-side repo_locking win out if allowed": {
@@ -957,6 +964,7 @@ repos:
ApplyRequirements: []string{},
ImportRequirements: []string{},
RepoLocking: Bool(true),
CustomPolicyCheck: Bool(false),
},
repoWorkflows: nil,
exp: valid.MergedProjectCfg{
@@ -970,6 +978,7 @@ repos:
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: false,
CustomPolicyCheck: false,
},
},
"last server-side match wins": {
@@ -1006,6 +1015,7 @@ repos:
AutoplanEnabled: false,
PolicySets: emptyPolicySets,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"autoplan is set properly": {
@@ -1032,6 +1042,7 @@ repos:
AutoplanEnabled: true,
PolicySets: emptyPolicySets,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
"execution order group is set": {
@@ -1060,6 +1071,7 @@ repos:
PolicySets: emptyPolicySets,
ExecutionOrderGroup: 10,
RepoLocking: true,
CustomPolicyCheck: false,
},
},
}
@@ -1254,6 +1266,7 @@ repos:
PolicySets: emptyPolicySets,
RepoLocking: true,
PolicyCheck: false,
CustomPolicyCheck: false,
},
},
"global policy check enabled": {
@@ -1293,6 +1306,7 @@ repos:
PolicySets: emptyPolicySets,
RepoLocking: true,
PolicyCheck: true,
CustomPolicyCheck: false,
},
},
"global policy check enabled except current repo": {
@@ -1333,6 +1347,7 @@ repos:
PolicySets: emptyPolicySets,
RepoLocking: true,
PolicyCheck: false,
CustomPolicyCheck: false,
},
},
"global policy check disabled and disabled on current repo": {
@@ -1373,6 +1388,7 @@ repos:
PolicySets: emptyPolicySets,
RepoLocking: true,
PolicyCheck: false,
CustomPolicyCheck: false,
},
},
"global policy check disabled and enabled on current repo": {
@@ -1413,6 +1429,7 @@ repos:
PolicySets: emptyPolicySets,
RepoLocking: true,
PolicyCheck: true, // Project will have policy check as true but since it is globally disable it wont actually run
CustomPolicyCheck: false,
},
},
}

2
server/core/config/valid/repo_cfg.go
View File

@@ -24,6 +24,7 @@ type RepoCfg struct {
ParallelPolicyCheck *bool
DeleteSourceBranchOnMerge *bool
RepoLocking *bool
CustomPolicyCheck *bool
EmojiReaction string
AllowedRegexpPrefixes []string
AbortOnExcecutionOrderFail bool
@@ -136,6 +137,7 @@ type Project struct {
RepoLocking *bool
ExecutionOrderGroup int
PolicyCheck *bool
CustomPolicyCheck *bool
}
// GetName returns the name of the project or an empty string if there is no

8
server/core/runtime/policy/conftest_client.go
View File

@@ -208,10 +208,10 @@ func (c *ConfTestExecutorWorkflow) Run(ctx command.ProjectContext, executablePat
}
policySetResults = append(policySetResults, models.PolicySetResult{
PolicySetName: policySet.Name,
ConftestOutput: cmdOutput,
Passed: passed,
ReqApprovals: policySet.ApproveCount,
PolicySetName: policySet.Name,
PolicyOutput: cmdOutput,
Passed: passed,
ReqApprovals: policySet.ApproveCount,
})
}

10
server/core/runtime/policy/conftest_client_test.go
View File

@@ -191,7 +191,7 @@ func TestRun(t *testing.T) {
var extraArgs []string
expectedOutput := "Success"
expectedResult := `[{"PolicySetName":"policy1","ConftestOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","ConftestOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedResult := `[{"PolicySetName":"policy1","PolicyOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","PolicyOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedArgsPolicy1 := []string{executablePath, "test", "-p", localPolicySetPath1, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
expectedArgsPolicy2 := []string{executablePath, "test", "-p", localPolicySetPath2, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
@@ -216,7 +216,7 @@ func TestRun(t *testing.T) {
extraArgs := []string{"--all-namespaces"}
expectedOutput := "Success"
expectedResult := `[{"PolicySetName":"policy1","ConftestOutput":"","Passed":true,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","ConftestOutput":"","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedResult := `[{"PolicySetName":"policy1","PolicyOutput":"","Passed":true,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","PolicyOutput":"","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedArgsPolicy1 := []string{executablePath, "test", "-p", localPolicySetPath1, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
expectedArgsPolicy2 := []string{executablePath, "test", "-p", localPolicySetPath2, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
@@ -241,7 +241,7 @@ func TestRun(t *testing.T) {
var extraArgs []string
expectedOutput := "Success"
expectedResult := `[{"PolicySetName":"policy1","ConftestOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedResult := `[{"PolicySetName":"policy1","PolicyOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedArgsPolicy1 := []string{executablePath, "test", "-p", localPolicySetPath1, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
expectedArgsPolicy2 := []string{executablePath, "test", "-p", localPolicySetPath2, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
@@ -284,7 +284,7 @@ func TestRun(t *testing.T) {
expectedOutputPolicy1 := fmt.Sprintf("FAIL - %s - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions", filepath.Join(workdir, "testproj-default.json"))
expectedOutputPolicy2 := "Success"
expectedResult := `[{"PolicySetName":"policy1","ConftestOutput":"FAIL - <redacted plan file> - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions","Passed":false,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","ConftestOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedResult := `[{"PolicySetName":"policy1","PolicyOutput":"FAIL - <redacted plan file> - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions","Passed":false,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","PolicyOutput":"Success","Passed":true,"ReqApprovals":0,"CurApprovals":0}]`
expectedArgsPolicy1 := []string{executablePath, "test", "-p", localPolicySetPath1, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
expectedArgsPolicy2 := []string{executablePath, "test", "-p", localPolicySetPath2, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
@@ -306,7 +306,7 @@ func TestRun(t *testing.T) {
var extraArgs []string
expectedOutput := fmt.Sprintf("FAIL - %s - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions", filepath.Join(workdir, "testproj-default.json"))
expectedResult := `[{"PolicySetName":"policy1","ConftestOutput":"FAIL - <redacted plan file> - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions","Passed":false,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","ConftestOutput":"FAIL - <redacted plan file> - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions","Passed":false,"ReqApprovals":0,"CurApprovals":0}]`
expectedResult := `[{"PolicySetName":"policy1","PolicyOutput":"FAIL - <redacted plan file> - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions","Passed":false,"ReqApprovals":0,"CurApprovals":0},{"PolicySetName":"policy2","PolicyOutput":"FAIL - <redacted plan file> - failure\n1 tests, 0 passed, 0 warnings, 1 failure, 0 exceptions","Passed":false,"ReqApprovals":0,"CurApprovals":0}]`
expectedArgsPolicy1 := []string{executablePath, "test", "-p", localPolicySetPath1, filepath.Join(workdir, "testproj-default.json"), "--no-color"}
expectedArgsPolicy2 := []string{executablePath, "test", "-p", localPolicySetPath2, filepath.Join(workdir, "testproj-default.json"), "--no-color"}

8
server/core/runtime/run_step_runner.go
View File

@@ -73,8 +73,12 @@ func (r *RunStepRunner) Run(ctx command.ProjectContext, command string, path str
if err != nil {
err = fmt.Errorf("%s: running %q in %q: \n%s", err, command, path, output)
ctx.Log.Debug("error: %s", err)
return "", err
if !ctx.CustomPolicyCheck {
ctx.Log.Debug("error: %s", err)
return "", err
} else {
ctx.Log.Debug("Treating custom policy tool error exit code as a policy failure. Error output: %s", err)
}
}
switch postProcessOutput {

2
server/events/command/project_context.go
View File

@@ -122,6 +122,8 @@ type ProjectContext struct {
ExecutionOrderGroup int
// If plans/applies should be aborted if any prior plan/apply fails
AbortOnExcecutionOrderFail bool
// Allows custom policy check tools outside of Conftest to run in checks
CustomPolicyCheck bool
}
// SetProjectScopeTags adds ProjectContext tags to a new returned scope.

42
server/events/markdown_renderer_test.go
View File

@@ -267,7 +267,7 @@ $$$
{
PolicySetName: "policy1",
// strings.Repeat require to get wrapped result
ConftestOutput: `FAIL - <redacted plan file> - main - WARNING: Null Resource creation is prohibited.
PolicyOutput: `FAIL - <redacted plan file> - main - WARNING: Null Resource creation is prohibited.
2 tests, 1 passed, 0 warnings, 1 failure, 0 exceptions`,
Passed: false,
@@ -276,9 +276,9 @@ $$$
{
PolicySetName: "policy2",
// strings.Repeat require to get wrapped result
ConftestOutput: "2 tests, 2 passed, 0 warnings, 0 failure, 0 exceptions",
Passed: true,
ReqApprovals: 1,
PolicyOutput: "2 tests, 2 passed, 0 warnings, 0 failure, 0 exceptions",
Passed: true,
ReqApprovals: 1,
},
},
LockURL: "lock-url",
@@ -335,7 +335,7 @@ $$$
{
PolicySetName: "policy1",
// strings.Repeat require to get wrapped result
ConftestOutput: strings.Repeat("line\n", 13) + `FAIL - <redacted plan file> - main - WARNING: Null Resource creation is prohibited.
PolicyOutput: strings.Repeat("line\n", 13) + `FAIL - <redacted plan file> - main - WARNING: Null Resource creation is prohibited.
2 tests, 1 passed, 0 warnings, 1 failure, 0 exceptions`,
Passed: false,
@@ -565,9 +565,9 @@ $$$
PolicyCheckResults: &models.PolicyCheckResults{
PolicySetResults: []models.PolicySetResult{
models.PolicySetResult{
PolicySetName: "policy1",
ConftestOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
PolicySetName: "policy1",
PolicyOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
},
},
LockURL: "lock-url",
@@ -582,9 +582,9 @@ $$$
PolicyCheckResults: &models.PolicyCheckResults{
PolicySetResults: []models.PolicySetResult{
models.PolicySetResult{
PolicySetName: "policy1",
ConftestOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
PolicySetName: "policy1",
PolicyOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
},
}, LockURL: "lock-url2",
ApplyCmd: "atlantis apply -d path2 -w workspace",
@@ -778,9 +778,9 @@ $$$
PolicyCheckResults: &models.PolicyCheckResults{
PolicySetResults: []models.PolicySetResult{
models.PolicySetResult{
PolicySetName: "policy1",
ConftestOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
PolicySetName: "policy1",
PolicyOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
},
}, LockURL: "lock-url",
ApplyCmd: "atlantis apply -d path -w workspace",
@@ -794,10 +794,10 @@ $$$
PolicyCheckResults: &models.PolicyCheckResults{
PolicySetResults: []models.PolicySetResult{
models.PolicySetResult{
PolicySetName: "policy1",
ConftestOutput: "4 tests, 2 passed, 0 warnings, 2 failures, 0 exceptions",
Passed: false,
ReqApprovals: 1,
PolicySetName: "policy1",
PolicyOutput: "4 tests, 2 passed, 0 warnings, 2 failures, 0 exceptions",
Passed: false,
ReqApprovals: 1,
},
}, LockURL: "lock-url",
ApplyCmd: "atlantis apply -d path -w workspace",
@@ -1317,9 +1317,9 @@ func TestRenderCustomPolicyCheckTemplate_DisableApplyAll(t *testing.T) {
PolicyCheckResults: &models.PolicyCheckResults{
PolicySetResults: []models.PolicySetResult{
models.PolicySetResult{
PolicySetName: "policy1",
ConftestOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
PolicySetName: "policy1",
PolicyOutput: "4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions",
Passed: true,
},
}, LockURL: "lock-url",
ApplyCmd: "atlantis apply -d path -w workspace",

14
server/events/models/models.go
View File

@@ -368,11 +368,11 @@ type PlanSuccess struct {
}
type PolicySetResult struct {
PolicySetName string
ConftestOutput string
Passed bool
ReqApprovals int
CurApprovals int
PolicySetName string
PolicyOutput string
Passed bool
ReqApprovals int
CurApprovals int
}
// PolicySetApproval tracks the number of approvals a given policy set has.
@@ -472,7 +472,7 @@ func (p *PolicyCheckResults) CombinedOutput() string {
combinedOutput := ""
for _, psResult := range p.PolicySetResults {
// accounting for json output from conftest.
for _, psResultLine := range strings.Split(psResult.ConftestOutput, "\\n") {
for _, psResultLine := range strings.Split(psResult.PolicyOutput, "\\n") {
combinedOutput = fmt.Sprintf("%s\n%s", combinedOutput, psResultLine)
}
}
@@ -484,7 +484,7 @@ func (p *PolicyCheckResults) Summary() string {
note := ""
for _, policySetResult := range p.PolicySetResults {
r := regexp.MustCompile(`\d+ tests?, \d+ passed, \d+ warnings?, \d+ failures?, \d+ exceptions?(, \d skipped)?`)
if match := r.FindString(policySetResult.ConftestOutput); match != "" {
if match := r.FindString(policySetResult.PolicyOutput); match != "" {
note = fmt.Sprintf("%s\npolicy set: %s: %s", note, policySetResult.PolicySetName, match)
}
}

16
server/events/models/models_test.go
View File

@@ -433,8 +433,8 @@ func TestPolicyCheckResults_Summary(t *testing.T) {
description: "test single format with single policy set",
policysetResults: []models.PolicySetResult{
{
PolicySetName: "policy1",
ConftestOutput: "20 tests, 19 passed, 2 warnings, 0 failures, 0 exceptions",
PolicySetName: "policy1",
PolicyOutput: "20 tests, 19 passed, 2 warnings, 0 failures, 0 exceptions",
},
},
exp: "policy set: policy1: 20 tests, 19 passed, 2 warnings, 0 failures, 0 exceptions",
@@ -443,16 +443,16 @@ func TestPolicyCheckResults_Summary(t *testing.T) {
description: "test multiple formats with multiple policy sets",
policysetResults: []models.PolicySetResult{
{
PolicySetName: "policy1",
ConftestOutput: "20 tests, 19 passed, 2 warnings, 0 failures, 0 exceptions",
PolicySetName: "policy1",
PolicyOutput: "20 tests, 19 passed, 2 warnings, 0 failures, 0 exceptions",
},
{
PolicySetName: "policy2",
ConftestOutput: "3 tests, 0 passed, 1 warning, 1 failure, 0 exceptions, 1 skipped",
PolicySetName: "policy2",
PolicyOutput: "3 tests, 0 passed, 1 warning, 1 failure, 0 exceptions, 1 skipped",
},
{
PolicySetName: "policy3",
ConftestOutput: "1 test, 0 passed, 1 warning, 1 failure, 1 exception",
PolicySetName: "policy3",
PolicyOutput: "1 test, 0 passed, 1 warning, 1 failure, 1 exception",
},
},
exp: `policy set: policy1: 20 tests, 19 passed, 2 warnings, 0 failures, 0 exceptions

1
server/events/project_command_context_builder.go
View File

@@ -274,6 +274,7 @@ func newProjectCommandContext(ctx *command.Context,
AutomergeEnabled: automergeEnabled,
DeleteSourceBranchOnMerge: projCfg.DeleteSourceBranchOnMerge,
RepoLocking: projCfg.RepoLocking,
CustomPolicyCheck: projCfg.CustomPolicyCheck,
ParallelApplyEnabled: parallelApplyEnabled,
ParallelPlanEnabled: parallelPlanEnabled,
ParallelPolicyCheckEnabled: parallelPlanEnabled,

19
server/events/project_command_runner.go
View File

@@ -425,7 +425,7 @@ func (p *DefaultProjectCommandRunner) doPolicyCheck(ctx command.ProjectContext)
if !lockAttempt.LockAcquired {
return nil, lockAttempt.LockFailureReason, nil
}
ctx.Log.Debug("acquired lock for project")
ctx.Log.Debug("acquired lock for project.")
// Acquire internal lock for the directory we're going to operate in.
// We should refactor this to keep the lock for the duration of plan and policy check since as of now
@@ -489,14 +489,23 @@ func (p *DefaultProjectCommandRunner) doPolicyCheck(ctx command.ProjectContext)
var preConftestOutput []string
var postConftestOutput []string
var policySetResults []models.PolicySetResult
for i, output := range outputs {
index = i
err = json.Unmarshal([]byte(strings.Join([]string{output}, "\n")), &policySetResults)
if err == nil {
break
if !ctx.CustomPolicyCheck {
err = json.Unmarshal([]byte(strings.Join([]string{output}, "\n")), &policySetResults)
if err == nil {
break
}
preConftestOutput = append(preConftestOutput, output)
} else {
// Using a policy tool other than Conftest, manually building result struct
passed := !strings.Contains(strings.ToLower(output), "fail")
policySetResults = append(policySetResults, models.PolicySetResult{PolicySetName: "Custom", PolicyOutput: output, Passed: passed, ReqApprovals: 1, CurApprovals: 0})
preConftestOutput = append(preConftestOutput, "")
}
preConftestOutput = append(preConftestOutput, output)
}
if policySetResults == nil {
return nil, "", errors.New("unable to unmarshal conftest output")
}

2
server/events/templates/policy_check.tmpl
View File

@@ -3,7 +3,7 @@
{{ range $ps, $policy_sets }}
#### Policy Set: `{{ $ps.PolicySetName }}`
```diff
{{ $ps.ConftestOutput }}
{{ $ps.PolicyOutput }}
```
{{ end }}
{{ end }}