IaC Scan Runner

The IaC Scan Runner is a REST API service used to scan IaC (Infrastructure as Code) package by performing various checks in order to find possible errors and improvements. The IaC Scan Runner’s source code is available in xlab-si/iac-scan-runner GitHub repository.

Scanner and check reference

The scanner is the main component of the IaC Scan Runner and it initiates the scanning process, which makes the supplied IaC go through multiple checks.

IaC Scan Runner currently supports the following IaC checks that can be executed as part of one IaC scan:

IaC Check

Target IaC entity

Enabled (by default)

Needs configuration

xOpera TOSCA parser

TOSCA

yes

no

Ansible Lint

Ansible

yes

no

Steampunk Scanner

Ansible

no

yes

TFLint

Terraform

yes

no

tfsec

Terraform

yes

no

Terrascan

Terraform

yes

no

yamllint

YAML

yes

no

Pylint

Python

yes

no

Bandit

Python

yes

no

Safety

Python packages

yes

no

Gitleaks

Git repositories

yes

no

git-secrets

Git repositories

yes

no

Markdown lint

Markdown files

yes

no

hadolint

Docker

yes

no

Gixy

Nginx configuration

yes

no

ShellCheck

Shell scripts

yes

no

ESLint

JavaScript

yes

no

TypeScript ESLint

TypeScript

yes

no

HTMLHint

HTML

yes

no

stylelint

CSS and other styles

yes

no

Checkstyle

Java

yes

no

cloc

Multiple components

yes

no

Snyk

Multiple components

no

yes

SonarScanner

Multiple components

no

yes

The following subsections explain the necessary API actions for each check.


xOpera TOSCA validation

xOpera is an orchestration project that includes xOpera TOSCA orchestrator called opera - a lightweight TOSCA orchestrator, which uses a separate component called xOpera TOSCA YAML parser. This opera-tosca-parser Python library (also comes with CLI) is able to parse and validate OASIS TOSCA templates and CSAR files (also see xOpera validate check).

Check ID (from the API)

opera-tosca-parser

Enabled (by default)

yes

Configured (by default)

yes

Documentation

xOpera docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Not supported.

Secret

Not supported.


Ansible Lint

Ansible Lint is a command-line tool for linting playbooks, roles and collections aimed towards any Ansible users (see Ansible Lint check).

Check ID (from the API)

ansible-lint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Ansible Lint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional YAML configuration file (see Ansible Lint config). You can also skip this configuration and put your local configuration file called .ansible-lint to the root of your IaC package.

Secret

Not supported.


Steampunk Scanner

Steampunk Scanner (developed by XLAB Steampunk) is a quality scanner for Ansible playbooks, where you can scan the playbook and get an instant quality score with tips on how to improve it. The steampunk-scanner CLI enables the use of Ansible scanner from the console with the ability to scan Ansible task files, playbooks, roles and collections (see Steampunk Scanner check).

Check ID (from the API)

steampunk-scanner

Enabled (by default)

no

Configured (by default)

no

Documentation

Steampunk Scanner docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Not supported.

Secret

Accepts username (e-mail) and user token (inputted as <username>:<token>) needed to authenticate to Steampunk Scanner API. The API token has to be first generated by downloading steampunk-scanner package and the by running steampunk-scanner authenticate CLI command that will register the new user and send the token to the user’s e-mail.


TFLint

TFLint is a a pluggable Terraform linter.

Check ID (from the API)

tflint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

TFLint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional HCL configuration file (see TFLint config). You can also skip this configuration and put the TFLint config file named .tflint.hcl to the root of your IaC package.

Secret

Not supported.


tfsec

tfsec is a security scanner for your Terraform code (see tfsec check).

Check ID (from the API)

tfsec

Enabled (by default)

yes

Configured (by default)

yes

Documentation

tfsec docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional JSON or YAML configuration file (see tfsec config). You can also skip this configuration and put the tfsec config in the .tfsec folder in the IaC root and name it config.json or config.yml and it will be automatically loaded and used.

Secret

Not supported.


Terrascan

Terrascan is a static code analyzer for IaC and defaults to scanning Terraform (see Terrascan check).

Check ID (from the API)

terrascan

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Terrascan docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional TOML configuration file (see Terrascan config).

Secret

Not supported.


yamllint

yamllint is a linter for YAML files that checks for syntax validity, key repetition and cosmetic problems such as lines length, trailing spaces, indentation, etc. (see yamllint check).

Check ID (from the API)

yamllint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

yamllint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional YAML configuration file (see yamllint config). You can also skip the configuration put the configuration file to the root of your IaC package.

Secret

Not supported.


Pylint

Pylint is a Python static code analysis tool that checks for errors in Python code, tries to enforce a coding standard and looks for code smells (see Pylint check).

Check ID (from the API)

pylint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Pylint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional TOML configuration file (see Pylint config). You can also skip this configuration and put the config file (it could be called .pylintrc or there are numerous other options).

Secret

Not supported.


Bandit

Bandit is a tool designed to find common security issues in Python code (see Bandit check).

Check ID (from the API)

bandit

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Bandit docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional YAML or TOML configuration file (see Bandit config).

Secret

Not supported.


Safety

Safety is a is a PyUp CLI tool that checks your installed Python dependencies for known security vulnerabilities (see PyUp Safety check).

Check ID (from the API)

pyup-safety

Enabled (by default)

yes

Configured (by default)

yes

Documentation

PyUp Safety docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Not supported.

Secret

Not supported.


Gitleaks

Gitleaks is a SAST tool for detecting hardcoded secrets like passwords, API keys, and tokens in Git repos (see Gitleaks check).

Check ID (from the API)

git-leaks

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Gitleaks docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional TOML configuration file (see Gitleaks config).

Secret

Not supported.


git-secrets

git-secrets is a tool that prevents you from committing secrets and credentials into Git repositories (see git-secrets check).

Check ID (from the API)

git-secrets

Enabled (by default)

yes

Configured (by default)

yes

Documentation

git-secrets docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Not supported.

Secret

Not supported.


Markdown lint

Markdown lint is a tool to check markdown files and flag style issues (see Markdown lint check).

Check ID (from the API)

markdown-lint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Markdown lint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional .rc or .mdlrc configuration file (see Markdown lint config). You can also skip the configuration put the configuration file named .mdlrc to the root of your IaC package.

Secret

Not supported.


hadolint

hadolint is a Dockerfile linter (see hadolint check).

Check ID (from the API)

hadolint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

hadolint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional YAML configuration file (see hadolint config). You can also skip this configuration and put the configuration file (with the name .hadolint.yaml or .hadolint.yml) to the root of your IaC package.

Secret

Not supported.


Gixy

Gixy is a tool to analyze Nginx configuration (see Gixy check).

Check ID (from the API)

gixy

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Gixy docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional .conf configuration file (see Gixy config).

Secret

Not supported.


ShellCheck

stylelint is a static analysis tool for shell scripts (see ShellCheck check).

Check ID (from the API)

shellcheck

Enabled (by default)

yes

Configured (by default)

yes

Documentation

ShellCheck docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Not supported.

Secret

Not supported.


ESLint

ESLint is a tool for identifying and reporting on patterns found in ECMAScript/JavaScript code (see ESLint check).

Check ID (from the API)

es-lint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

ESLint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional configuration file (see ESLint config). You can also skip this configuration and put the configuration file to the root of your IaC package.

Secret

Not supported.


TypeScript ESLint

TypeScript ESLint enables ESLint to support TypeScript (see TypeScript ESLint check).

Check ID (from the API)

ts-lint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

TypeScript ESLint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional configuration file (see TypeScript ESLint config). You can also skip this configuration and put the configuration file to the root of your IaC package.

Secret

Not supported.


HTMLHint

HTMLHint is the static code analysis tool you need for your HTML (see HTMLHint check).

Check ID (from the API)

htmlhint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

HTMLHint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional .conf configuration file (see HTMLHint config). You can also skip this configuration and put the configuration file called .htmlhintrc to the root of your IaC package.

Secret

Not supported.


stylelint

stylelint is a mighty, modern linter that helps you avoid errors and enforce conventions in your styles (see stylelint check).

Check ID (from the API)

stylelint

Enabled (by default)

yes

Configured (by default)

yes

Documentation

stylelint docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional configuration file (see stylelint config). You can also skip this configuration and put the configuration file called to the root of your IaC package.

Secret

Not supported.


Checkstyle

Checkstyle is a tool for checking Java source code for adherence to a Code Standard or set of validation rules (see Checkstyle check).

Check ID (from the API)

checkstyle

Enabled (by default)

yes

Configured (by default)

yes

Documentation

Checkstyle docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional XML configuration file (see Checkstyle config).

Secret

Not supported.


cloc

cloc counts blank lines, comment lines, and physical lines of source code in many programming languages (see cloc check).

Check ID (from the API)

cloc

Enabled (by default)

yes

Configured (by default)

yes

Documentation

cloc docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Accepts an optional .txt configuration file (see cloc config).

Secret

Not supported.


Snyk

Snyk helps you find, fix and monitor known vulnerabilities in open source (see Snyk check).

Check ID (from the API)

snyk

Enabled (by default)

no

Configured (by default)

no

Documentation

Snyk docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Not supported.

Secret

Requires user to pass his API token to authenticate to Snyk. The API token is generated in the Snyk UI user settings and requires user to set up Snyk account (for more info see Snyk API token).


SonarScanner

SonarScanner is the official scanner used to run code analysis on SonarQube and SonarCloud (see SonarScanner check).

Check ID (from the API)

sonar-scanner

Enabled (by default)

no

Configured (by default)

no

Documentation

SonarScanner docs

Configuration options for /checks/{check_name}/configure API endpoint

Config file

Requires to pass a configuration file (see SonarScanner config), where you must specify your SonarQube analysis parameters. You might also have to create a new organization and project with the proper permissions in the UI.

Secret

Accepts an optional user token to authenticate to SonarQube or SonarCloud. The API token is generated in the SonarQube or SonarCloud UI user settings and requires user to have an account (for more info see SonarQube user authentication token).

Tip

If you do not wish to supply your user token within the config file, pass it as a secret in the API.


REST API

This section focuses on the IaC Scan Runner REST API service.

Installation

You can run the REST API using a public xscanner/runner Docker image as follows:

# run IaC Scan Runner REST API in a Docker container and
# navigate to localhost:8080/swagger or localhost:8080/redoc
docker run --name iac-scan-runner -p 8080:80 xscanner/runner

Tip

Other methods of running are also explained in xlab-si/iac-scan-runner GitHub repository.

Note

The Docker image is large because we use many different tools for scanning. For the future releases. we will try to shrink it down as much as possible.

Usage

After the setup you will see that the OpenAPI Specification and interactive Swagger UI API documentation are available on /swagger, whereas ReDoc generated API reference documentation is accessible on /redoc. You can also retrieve an OpenAPI document that conforms to the OpenAPI Specification as JSON file on /openapi.json or as YAML file on /openapi.yaml (or /openapi.yml).

The IaC Scan Runner API can be used to interact with the main IaC inspection component and initialize IaC scans. The API includes various IaC checks that can be filtered and configured. User can choose to execute all or just the selected checks as a part of one IaC scan. After the scanning process the API will return all the check results.

REST API endpoint

Description

/checks

Retrieve and filter checks

/checks/{check_name}/enable

Enable check for running

/checks/{check_name}/disable

Disable check for running

/checks/{check_name}/configure

Configure check

/scan

Initiate IaC scan

The API endpoints are further described below.


GET /checks

This endpoint lets you retrieve and filter the supported IaC checks. You can filter checks by their keynames (use the keyword request parameter) and find out whether they are already enabled (set the enabled parameter) or configured (set the configured parameter). Checks can also be filtered by their target entity (set the target_entity_type parameter) - here we have three types of checks - IaC (they only check the code), component (they check IaC requirements and dependencies in order to find vulnerabilities) and check that are both IaC and component. Each IaC check in the API has its unique name so that it can be distinguished from other checks. When no filter is specified, the endpoint lists all IaC checks.

Example request:

$ curl -X 'GET' 'http://127.0.0.1:8000/checks?keyword=Terraform&enabled=true'

Example response:

[
  {
    "name": "tflint",
    "description": "A Pluggable Terraform Linter",
    "enabled": true,
    "configured": true,
    "target_entity_type": "IaC"
  },
  {
    "name": "tfsec",
    "description": "Security scanner for your Terraform code",
    "enabled": true,
    "configured": true,
    "target_entity_type": "IaC"
  },
  {
    "name": "terrascan",
    "description": "Terrascan is a static code analyzer for IaC (defaults to scanning Terraform)",
    "enabled": true,
    "configured": true,
    "target_entity_type": "IaC"
  }
]
Query Parameters
  • keyword (string) – optional keyword from check name or description

  • enabled (boolean) – search for checks that are enabled or not

  • configured (string) – search for checks that are configured or not

  • target_entity_type (string) – search by target entity (one of IaC, component, IaC and component)

Status Codes

PUT /checks/{check_name}/enable

IaC checks can be enabled (can be used for scanning) or disabled (cannot be used for scanning). Most of the local checks are enabled by default and some of them that are advanced, take longer time or require additional configuration are disabled and have to be enabled before the scanning. This endpoint can be used to enable a specific IaC check (selected by the check_name parameter), which means that it will become available for running within IaC scans.

Example request:

$ curl -X 'PUT' 'http://127.0.0.1:8000/checks/snyk/enable'

Example response:

"Check: snyk is now enabled and available to use."
Parameters
  • check_name (string) – check that you want to enable for running

Status Codes

PUT /checks/{check_name}/disable

This endpoint can be used to disable a specific IaC check (selected by the check_name parameter), which means that it will become unavailable for running within IaC scans.

Example request:

$ curl -X 'PUT' 'http://127.0.0.1:8000/checks/pylint/disable'

Example response:

"Check: pylint is now disabled and cannot be used."
Parameters
  • check_name (string) – check that you want to disable for running

Status Codes

PUT /checks/{check_name}/configure

This endpoint is used to configure a specific IaC check (selected by the check_name parameter). Most IaC checks do not need configuration as they already use their default settings. However, some of them - especially the remote service checks (such as Snyk) require to be configured before using them within IaC scans. Some checks will have to be enabled before they can be configured. The configuration of IaC check takes two optional multipart request body parameters - config_file and secret. The former (config_file) can be used to pass a check configuration file (which is supported by almost every check) that is specific to every check and will override the default check settings. The latter (secret) is meant for passing sensitive data such as passwords, API keys, tokens, etc. These secrets are often used to configure the remote service checks - usually to authenticate the user via some token that has been generated in the remote service user profile settings. Some IaC checks support both the aforementioned request body parameters and some support one of them or none. The API will warn you in case of any configuration problems.

Example request:

$ curl -X 'PUT' 'http://127.0.0.1:8000/checks/sonar-scanner/configure' -H 'Content-Type: multipart/form-data' -F 'config_file=@sonar-project.properties;type=text/plain' -F 'secret=56bf-example-token-f007'

Example response:

"Check: sonar-scanner has been configured successfully."
Parameters
  • check_name (string) – check that you want to configure before scanning

Form Parameters
  • config_file – optional check configuration file

  • secret – optional secret for configuration (password, API token, etc.)

Status Codes

Warning

Be careful not to expose your secrets directly in your IaC.


POST /scan

This is the main endpoint that is used to scan the IaC and gather the results from the executed IaC checks. The request body is treated as multipart (multipart/form-data type) and has two parameters. The first one is iac and is required. Here, the user passes his (compressed) IaC package (currently limited to zip or tar). The second parameter is checks and is an optional array of checks, which the user wants to executed as a part of his IaC scan. The IaC checks are selected by their unique names. If the user does not specify that field, all the enabled checks are executed. The API will warn you if there are any nonexistent, disabled or un-configured checks that you wanted to use. After the scanning process the API will return results of all checks (their outputs and return codes).

Example request:

$ curl -X 'POST' 'http://127.0.0.1:8000/scan' -H 'Content-Type: multipart/form-data' -F 'iac=@scaling-example.zip' -F 'checks=bandit,ansible-lint'

Example response:

{
  "bandit": {
    "output": "[main]\tINFO\tprofile include tests: None\n[main]\tINFO\tprofile exclude tests: None\n[main]\tINFO\tcli include tests: None\n[main]\tINFO\tcli exclude tests: None\n[main]\tINFO\trunning on Python 3.8.10\nRun started:2021-08-25 11:23:29.960356\n\nTest results:\n\tNo issues identified.\n\nCode scanned:\n\tTotal lines of code: 0\n\tTotal lines skipped (#nosec): 0\n\nRun metrics:\n\tTotal issues (by severity):\n\t\tUndefined: 0\n\t\tLow: 0\n\t\tMedium: 0\n\t\tHigh: 0\n\tTotal issues (by confidence):\n\t\tUndefined: 0\n\t\tLow: 0\n\t\tMedium: 0\n\t\tHigh: 0\nFiles skipped (0):\n",
    "rc": 0
  },
  "ansible-lint": {
    "output": "WARNING  Listing 6 violation(s) that are fatal\n\u001b[34mservice.yaml\u001b[0m:32: \u001b[91myaml\u001b[0m \u001b[2mtoo many spaces inside braces\u001b[0m \u001b[2;91m(braces)\u001b[0m\n\u001b[34mservice.yaml\u001b[0m:32: \u001b[91myaml\u001b[0m \u001b[2mtoo many spaces inside brackets\u001b[0m \u001b[2;91m(brackets)\u001b[0m\n\u001b[34mservice.yaml\u001b[0m:35: \u001b[91myaml\u001b[0m \u001b[2mtoo many spaces inside braces\u001b[0m \u001b[2;91m(braces)\u001b[0m\n\u001b[34mservice.yaml\u001b[0m:35: \u001b[91myaml\u001b[0m \u001b[2mtoo many spaces inside brackets\u001b[0m \u001b[2;91m(brackets)\u001b[0m\n\u001b[34mservice.yaml\u001b[0m:45: \u001b[91myaml\u001b[0m \u001b[2mtoo many spaces inside brackets\u001b[0m \u001b[2;91m(brackets)\u001b[0m\n\u001b[34mservice.yaml\u001b[0m:62: \u001b[91myaml\u001b[0m \u001b[2mtoo many spaces inside brackets\u001b[0m \u001b[2;91m(brackets)\u001b[0m\nYou can skip specific rules or tags by adding them to your configuration file:\n\u001b[2m# .ansible-lint\u001b[0m\n\u001b[94mwarn_list\u001b[0m:  \u001b[2m# or 'skip_list' to silence them completely\u001b[0m\n  - yaml  \u001b[2m# Violations reported by yamllint\u001b[0m\n\nFinished with \u001b[1;36m6\u001b[0m \u001b[1;35mfailure\u001b[0m\u001b[1m(\u001b[0ms\u001b[1m)\u001b[0m, \u001b[1;36m0\u001b[0m \u001b[1;35mwarning\u001b[0m\u001b[1m(\u001b[0ms\u001b[1m)\u001b[0m on \u001b[1;36m9\u001b[0m files.\n",
    "rc": 2
  }
}
Parameters
  • scan_response_type (string) – JSON (default) or YAML scan response

Form Parameters
  • iac – IaC file (currently limited to zip or tar)

  • checks – optional array of the selected checks

Status Codes

Note

All API endpoints try to use JSON responses.

CLI

The IaC Scan Runner CLI enables easier setup of IaC Scan Runner in console environments.

Prerequisites

The Scan Runner CLI requires Python 3 and a virtual environment. In a typical modern Linux environment, we should already be set. In Ubuntu, however, we might need to run the following commands:

$ sudo apt update
$ sudo apt install -y python3-venv python3-wheel python-wheel-common

Installation

IaC Scan Runner CLI is distributed as Python iac-scan-runner package that is regularly published on PyPI. The simplest way to test iac-scan-runner is to install it into virtual environment:

$ mkdir ~/iac-scan-runner && cd ~/iac-scan-runner
$ python3 -m venv .venv && . .venv/bin/activate
(.venv) $ pip install --upgrade pip
(.venv) $ pip install iac-scan-runner

The development version of the package is available on TestPyPI and the installation goes as follows.

(.venv) $ pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ iac-scan-runner

Commands

iac-scan-runner currently allows users to execute the following shell commands:

CLI command

Purpose and description

iac-scan-runner openapi

print OpenAPI Specification

iac-scan-runner install

install the IaC Scan Runner prerequisites

iac-scan-runner run

run the IaC Scan Runner REST API

Tip

All the CLI commands are equipped with -h/--help option to help you.


iac-scan-runner

IaC Scan Runner CLI

iac-scan-runner [OPTIONS] COMMAND [ARGS]...

Options

--install-completion <install_completion>

Install completion for the specified shell.

Options

bash | zsh | fish | powershell | pwsh

--show-completion <show_completion>

Show completion for the specified shell, to copy it or customize the installation.

Options

bash | zsh | fish | powershell | pwsh

install

Install prerequisites for IaC Scan Runner

iac-scan-runner install [OPTIONS]
openapi

Get OpenAPI specification

iac-scan-runner openapi [OPTIONS]

Options

-f, --format <output_format>

OpenAPI output format

Default

OpenApiFormat.json

Options

json | yaml

-o, --output <output>

Output file path

run

Run REST API for IaC Scan Runner

iac-scan-runner run [OPTIONS]

Note

If you have any problems with IaC Scan Runner please have a look at the existing GitHub issues in xlab-si/iac-scan-runner/issues or open a new one yourself.