Go to Application

How to use the CircleCI local CLI

2 days ago · 21 min read

Cloud Server v4+

Contribute

This page describes how to use some features of the CircleCI CLI.

Prerequisites

To follow the guides on this page you will need the following:

A CircleCI account (you can sign up for free)
Local installation of the CircleCI CLI. Steps and options are available on the Installing the local CLI page.
Set up and configure the CircleCI CLI.

Validate a CircleCI config

Using VS Code? You can validate you CircleCI configuration using the VS Code extension. For full information about the features provided in the VS Code extension, see the VS Code extension overview page.

You can avoid pushing additional commits to test your .circleci/config.yml by using the CLI to validate your configuration locally. Follow these steps:

Navigate to your project directory. You can a directory with a .circleci/config.yml file
Run the following, providing the path to your .circleci/config.yml:
```
circleci config validate <path-to-config.yml>
```

If your configuration is valid, you should see the following:

Config file at .circleci/config.yml is valid

Otherwise you will see errors, for example:

Error: config compilation contains errors: config compilation contains errors:
	- Unable to parse YAML
	- mapping values are not allowed here
	-  in 'string', line 20, column 22:
	-               environment:
	-                          ^

Orb development kit

The orb development kit refers to a suite of tools that work together to simplify the orb development process, with automatic testing and deployment on CircleCI. There are two commands in the CLI that are a part of the orb development kit:

The following command initializes a new orb project:
```
circleci orb init
```
The following command packs an orb with local scripts:
```
circleci orb pack
```

For more information on orb packing, see the Orbs Concepts page.

Validate an orb in your configuration file

You can validate your orb with the following:

circleci orb validate <path-to-your-orb.yml>

Packing a config

circleci config pack

This CLI pack command (separate to circleci orb pack described above) allows you to create a single YAML file from several separate files (based on directory structure and file contents). The pack command implements FYAML, a scheme for breaking YAML documents across files in a directory tree. This is useful for breaking up source code for large orbs and allows custom organization of your orbs' YAML configuration.

How you name and organize your files when using the pack command will determine the final orb.yml output. Consider the following folder structure example:

When packing a config using circleci config pack, the component names are not included in the configuration files, they are taken from the file names.

$ tree
.
└── your-orb-source
    ├── @orb.yml
    ├── commands
    │   └── foo.yml
    └── jobs
        └── bar.yml

3 directories, 3 files

The UNIX tree command is great for printing out folder structures. In the example tree structure above, the pack command will map the folder names and file names to YAML keys, and map the file contents as the values to those keys.

The following command will pack up the example folder from above:

circleci config pack your-orb-source

And the output will be in your .yml file:

# Contents of @orb.yml appear here
commands:
  foo:
    # contents of foo.yml appear here
jobs:
  bar:
    # contents of bar.yml appear here

Other configuration packing capabilities

A file beginning with @ will have its contents merged into its parent folder level. This can be useful at the top level of an orb, when one might want generic orb.yml to contain metadata, but not to map into an orb key-value pair.

For example, the following structure:

cat foo/bar/@baz.yml
\{baz: qux}

Maps to:

bar:
  baz: qux

Processing a config

Running the following command validates your config, but will also display expanded source configuration alongside your original configuration. This is particularly useful if you are using orbs:

circleci config process <path-to-config.yml>

Consider the following example configuration that uses the node orb:

version: 2.1

orbs:
  node: circleci/node@4.7.0

workflows:
  example-workflow:
      jobs:
        - node/test

Processing this config file will output a YAML file like the example below. This is the expanded source configuration using version: 2 syntax. All version: 2.1 elements are processed:

# Orb 'circleci/node@4.7.0' resolved to 'circleci/node@4.7.0'
version: 2
jobs:
  node/test:
    docker:
    - image: cimg/node:13.11.0
    steps:
    - checkout
    - run:
        command: |
          if [ ! -f "package.json" ]; then
            echo
            echo "---"
            echo "Unable to find your package.json file. Did you forget to set the app-dir parameter?"
            echo "---"
            echo
            echo "Current directory: $(pwd)"
            echo
            echo
            echo "List directory: "
            echo
            ls
            exit 1
          fi
        name: Checking for package.json
        working_directory: ~/project
    - run:
        command: |
          if [ -f "package-lock.json" ]; then
            echo "Found package-lock.json file, assuming lockfile"
            ln package-lock.json /tmp/node-project-lockfile
          elif [ -f "npm-shrinkwrap.json" ]; then
            echo "Found npm-shrinkwrap.json file, assuming lockfile"
            ln npm-shrinkwrap.json /tmp/node-project-lockfile
          elif [ -f "yarn.lock" ]; then
            echo "Found yarn.lock file, assuming lockfile"
            ln yarn.lock /tmp/node-project-lockfile
          fi
          ln package.json /tmp/node-project-package.json
        name: Determine lockfile
        working_directory: ~/project
    - restore_cache:
        keys:
        - node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
        - node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-
        - node-deps-{{ arch }}-v1-{{ .Branch }}-
    - run:
        command: "if [[ ! -z \"\" ]]; then\n  echo \"Running override package installation command:\"\n  \nelse\n  npm ci\nfi\n"
        name: Installing NPM packages
        working_directory: ~/project
    - save_cache:
        key: node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
        paths:
        - ~/.npm
    - run:
        command: npm run test
        name: Run NPM Tests
        working_directory: ~/project
workflows:
  version: 2
  example-workflow:
    jobs:
    - node/test

Run a job in a container on your machine

The CircleCI CLI enables you to run a job from your configuration locally with Docker. This can be useful for the following:

To run tests before pushing configuration changes
Debugging your build process without impacting your build queue

Only single jobs can be run locally, not workflows.

Prerequisites

You will need to have Docker installed on your system, as well as the most recent version of the CLI. You will also need to have a project that includes a valid .circleci/config.yml file.

Run a job

Navigate to the root of your project containing the .circleci/config.yml file.
Run the following command specifying the job you would like to run. If your CircleCI configuration is set to version 2.1, you must first export your configuration to process.yml, and specify it when executing with the following commands:
```
circleci config process .circleci/config.yml > process.yml
circleci local execute -c process.yml <job-name>
```
If you are using version: 2 configuration, you can run:
```
circleci local execute <job-name>
```

The commands above will run the job you specify by name. The CLI uses Docker to pull down the requirements for the build and then execute your CI steps locally.

Limitations of running jobs locally

Although running jobs locally with circleci is helpful, there are some limitations.

Executors

The CLI does not support running jobs that use a machine (machine) or macOS (macos) executor locally. This is because these executors require running an additional virtual machine. Only jobs that use a Docker (docker) executor can be run locally.

Add SSH keys

It is currently not possible to add SSH keys using the add_ssh_keys CLI command.

Workflows

The CLI tool does not provide support for running workflows. By nature, workflows leverage running jobs concurrently on multiple machines allowing you to achieve faster, more complex builds. Because the CLI is only running on your machine, it can only run single jobs (which make up parts of a workflow).

Caching and online-only commands

Caching is not currently supported in local jobs. When you have either a save_cache or restore_cache step in your config, circleci will skip them and display a warning.

Further, not all commands may work on your local machine as they do online. For example, the Golang build reference above runs a store_artifacts step, however, local builds will not upload artifacts. If a step is not available on a local build you will see an error in the console.

Environment variables

For security reasons, encrypted environment variables configured in the web application will not be imported into local builds. As an alternative, you can specify environment variables to the CLI with the -e flag. See the output of the following command for more information.

circleci help build

If you have multiple environment variables, you must use the flag for each variable, for example:

circleci build -e VAR1=FOO -e VAR2=BAR

Test splitting

The CircleCI CLI is used for some advanced features during job runs, for example test splitting for build time optimization.

Project management

The CircleCI CLI allows you to execute several project-oriented actions.

Create a new project

This command only applies to GitLab or GitHub App organizations (Organization slug starts with circleci followed by a UUID. For example, circleci/34R3kN5RtfEE7v4sa4nWAU. You can find your organization slug under Organization Settings Overview).

Use the create sub-command to create a new project.

circleci project create <vcs-type> <org-id> [flags]

<vcs-type> depends on the type of integration for your organization. This command only supports the circleci VCS type.
<org-id> is the ID of the organization associated with the project. You can find it from the CircleCI web app under Organization Settings Overview.
--name is the flag that allows you to provide the project name. If not provided, you will be prompted to enter it.

For full details on the create sub-command, refer to the CLI docs.

Create an environment variable for a project

Use the secret create sub-command to create an environment variable for a project:

circleci project secret create <vcs-type> <org-id> <project-id> <env-name> [flags]

<vcs-type> depends on the type of integration for your organization. For projects that use GitLab or GitHub App, use circleci as vcs-type.
<org-id> is the ID of the organization associated with the project. You can find it from the CircleCI web app under Organization Settings Overview.
<project-id> is the ID of the project associated with the project. You can find it from the CircleCI web app under Project Settings Overview.
<env-name> is the name of the environment variable to create.
--env-value is the flag that allows you to provide the value of the environment variable. If not provided, you will be prompted to enter it.

For full details on the secret create sub-command, refer to the CLI docs.

List all environment variables for a project

Use the secret list sub-command to list all environment variables for a project:

circleci project secret list <vcs-type> <org-id> <project-id>

<vcs-type> depends on the type of integration for your organization. For projects that use GitLab or GitHub App, use circleci as vcs-type.
<org-id> is the ID of the organization associated with the project. You can find it from the CircleCI web app under Organization Settings Overview.
<project-id> is the ID of the project associated with the project. You can find it from the CircleCI web app under Project Settings Overview.

For full details on the secret list sub-command, refer to the CLI docs.

Purge Docker Layer Cache for a project

Purging the Docker Layer Cache means removing previously stored Docker image layers so that future builds will not reuse those cached layers. This forces CircleCI to rebuild Docker images from scratch, rather than leveraging layers from earlier builds to speed up the process.

Use the dlc purge sub-command to purge Docker Layer Cache for a project:

circleci dlc purge <vcs-type> <org-id> <project-id>

<vcs-type> depends on the type of integration for your organization. For projects that use GitLab or GitHub App, use circleci as vcs-type.
<org-id> is the ID of the organization associated with the project. You can find it from the CircleCI web app under Organization Settings Overview.
<project-id> is the ID of the project associated with the project. You can find it from the CircleCI web app under Project Settings Overview.

For full details on the dlc purge sub-command, refer to the CLI docs.

Pipeline management

The CircleCI CLI allows you to operate on pipelines. You can use the CLI to create a new pipeline definition, list existing pipeline definitions, or run a specific pipeline.

Create a new pipeline definition for a project

Use the create sub-command to create a new pipeline definition for your project.

The create sub-command of the pipeline command only applies to pipeline definitions created with the GitHub App provider Organization slug starts with circleci followed by a UUID. For example, circleci/34R3kN5RtfEE7v4sa4nWAU. You can find your organization slug under Organization Settings Overview).

circleci pipeline create <project-id> [flags]

<project-id> is the ID of the project for which you are creating the pipeline definition. You can find it from the CircleCI web app under Project Settings Overview.

The above example shows the minimal usage. You will be prompted for required values if you do not intially provide them using the appropriate flags. For example:

--repo-id is the flag that allows you to specify the ID of the GitHub repository you wish to build a pipeline definition for. It is required if the config file is in a different repository than the code repository. Refer to the GitHub documentation to find the ID.

For full details on the create sub-command, refer to the CLI docs.

List pipeline definitions for a project

Use the list sub-command to list all pipeline definitions for your project.

circleci pipeline list <project-id> [flags]

<project-id> is the ID of the project you wish to list the pipeline definitions for. You can find it from the CircleCI web app under Project Settings Overview.

For full details on the list sub-command, refer to the CLI docs.

Run a pipeline

Use the run sub-command to run a specific pipeline.

circleci pipeline run <org-id> <project-id> [flags]

<org-id> is the ID of the organization associated with the pipeline definition. You can find it from the CircleCI web app under Organization Settings Overview.
<project-id> is the ID of the project associated with the pipeline definition. You can find it from the CircleCI web app under Project Settings Overview.

The above example shows the minimal usage. You will be prompted for required values if you do not intially provide them using the appropriate flags. For example:

--pipeline-definition-id is the flag that allows you to provide the ID of the pipeline definition to run. If not provided, you will be prompted to enter it. You can retrieve it from the CircleCI web app by selecting Copy for the chosen pipeline, under Project settings Pipelines.

For full details on the run sub-command, refer to the CLI docs.

Create a trigger

You can use the CircleCI CLI to create triggers for your projects.

The trigger command and its create sub-command only apply to pipeline definitions created with the GitHub App provider (Organization slug starts with circleci followed by a UUID. For example, circleci/34R3kN5RtfEE7v4sa4nWAU. You can find your organization slug under Organization Settings Overview).

For full details on creating, testing, and managing triggers for your organization, see the Trigger a pipeline page.

Use the create sub-command to create a trigger for your project.

circleci trigger create <project-id>  [flags]

The above example shows the minimal usage. You will be prompted for required values if you do not intially provide them using the appropriate flags.

<project-id> is the ID of the project associated with the trigger. You can find it from the CircleCI web app under Project Settings Overview.
--pipeline-definition-id is the flag that allows you to provide the ID of the pipeline definition to run. If not provided, you will be prompted to enter it. You can retrieve it from the CircleCI web app by selecting Copy for the chosen pipeline, under Project settings Pipelines.
--repo-id is the flag that allows you to specify the ID of the GitHub repository you wish to build a pipeline definition for. It is required if the config file is in a different repository than the code repository. Refer to the GitHub documentation to find the ID.
--event-preset is the flag that allows you to specify the name of the event preset to use when filtering events for this trigger. The value can be any of:
- all-pushes
- only-tags
- default-branch-pushes
- only-build-prs
- only-open-prs
- only-labeled-prs
- only-merged-prs
- only-ready-for-review-prs
- only-branch-delete
- only-build-pushes-to-non-draft-prs

For full details on this command and sub-command, refer to the CLI docs.

Context management

Contexts provide a mechanism for securing and sharing environment variables across projects. While contexts have been traditionally managed on the CircleCI web application, the CircleCI CLI provides an alternative method for managing the usage of contexts in your projects. With the CLI, you can execute several context-oriented commands:

create - Create a new context
delete - Delete the named context
list - List all contexts
remove-secret - Remove an environment variable from the named context
show - Show a context
store-secret - Store a new environment variable in the named context

The above list are "sub-commands" in the CLI, which are executed like so:

circleci context create --org-id <org-id> <context-name> [flags]

Refer to the CLI docs for full details for each command. Many commands require that you include additional information as indicated by parameters delimited by < >. For example, when running circleci context create, you will need to provide a name for the context and your org ID.

To find your organization/user ID, select Organization Settings in the CircleCI web app side bar.

screenshot showing where to find your organization ID

Config policy management

The CircleCI CLI can be used to manage config policies for your projects. Config policies allow you to enforce best practices and standards across your organization.

For full details on creating, testing, and managing config policies for your organization, including how-to guides, see the Manage config policies section.

Next steps

Introduction to Execution Environments

[path]

[title]

How to use the CircleCI local CLI

Prerequisites

Validate a CircleCI config

Orb development kit

Validate an orb in your configuration file

Packing a config

Other configuration packing capabilities

Processing a config

Run a job in a container on your machine

Prerequisites

Run a job

Limitations of running jobs locally

Executors

Add SSH keys

Workflows

Caching and online-only commands

Environment variables

Test splitting

Project management

Create a new project

Create an environment variable for a project

List all environment variables for a project

Purge Docker Layer Cache for a project

Pipeline management

Create a new pipeline definition for a project

List pipeline definitions for a project

Run a pipeline

Create a trigger

Context management

Config policy management

Next steps