Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
625 lines (473 loc) · 24.4 KB

CONTRIBUTING.md

File metadata and controls

625 lines (473 loc) · 24.4 KB
Raw

Contributing to Conjur

Thanks for your interest in Conjur. Before contributing, please take a moment to read and sign our Contributor Agreement. This provides patent protection for all Conjur users and allows CyberArk to enforce its license terms. Please email a signed copy to oss@cyberark.com.

For general contribution and community guidelines, please see the community repo.

Table of contents generated with markdown-toc

Prerequisites

Before getting started, you should install some developer tools. These are not required to deploy Conjur but they will let you develop using a standardized, expertly configured environment.

  1. git to manage source code
  2. Docker to manage dependencies and runtime environments
  3. Docker Compose to orchestrate Docker environments
  4. Ruby version 3 or higher installed - native installation or using RVM.

Prevent Secret Leaks

Pushing to github is a form of publication, especially when using a public repo. It is a good idea to use a hook to check for secrets before pushing code. Follow this link to learn how to configure git checks for secrets before every push.

Build Conjur as a Docker image

It's easy to get started with Conjur and Docker:

  1. Install dependencies (as above)

  2. Clone this repository

  3. Run the build script in your terminal:

    $ ./build.sh
...
Successfully built 9a18a1396977
$ docker images | grep conjur
conjurinc/conjur latest a8229592474c 7 minutes ago 560.7 MB
conjur           latest a8229592474c 7 minutes ago 560.7 MB
conjur-dev       latest af98cb5b2a68 4 days ago    639.9 MB

Set up a development environment

The dev directory contains a docker-compose file which creates a development environment with a database container (pg, short for postgres), and a conjur server container with source code mounted into the directory /src/conjur-server.

To use it:

  1. Install dependencies (as above)

  2. Start the container (and optional extensions):

    $ cd dev
$ ./start
...
root@f39015718062:/src/conjur-server#

    Once the start script finishes, you're in a Bash shell inside the Conjur server container.

    After starting Conjur, your instance will be configured with the following:

    • Account: cucumber
    • User: admin
    • Password: Run conjurctl role retrieve-key cucumber:user:admin inside the container shell to retrieve the admin user API key (which is also the password)

  3. Run the server

    root@f39015718062:/src/conjur-server# conjurctl server
<various startup messages, then finally:>
* Listening on tcp://localhost:3000
Use Ctrl-C to stop

    The conjurctl server script performs the following:

    • wait for the database to be available
    • create and/or upgrade the database schema according to the db/migrate directory
    • find or create the token-signing key
    • start the web server

    You may choose to debug Conjur using pry.byebug or the Visual Studio Code IDE. This will allow you to work in the debugger without the server timing out. To do so, run the following command instead of conjurctl server:

  4. Cleanup

    $ ./stop

    Running stop removes the running Docker Compose containers and the data key.

LDAP Authentication

To enable a user to log into Conjur using LDAP credentials, run start with the --authn-ldap flag:

$ cd dev
$ ./start --authn-ldap
...
root@f39015718062:/src/conjur-server#

The --authn-ldap flag will:

  • Start an OpenLDAP container.
  • Load a user alice with the password alice into the LDAP server.
  • Load a policy authn-ldap/test, that grants alice the ability to authenticate via http://localhost:3000/authn-ldap/test/cucumber/alice/authenticate with the password alice.

Validate authentication using the username alice with the password alice:

$ curl -v -k -X POST -d "alice" http://localhost:3000/authn-ldap/test/cucumber/alice/authenticate

Google Cloud Platform (GCP) Authentication

To enable a host to log into Conjur using GCP identity token, run start with the --authn-gcp flag. Form more information on how to setup Conjur Google Cloud (GCP) authenticator, follow the official documentation.

Certificate Authentication with an Authenticator Service

To enable certificate authentication in Conjur, you need to have an authenticator service running - either as a container within Docker Compose or as a standalone service.

To start the authenticator service, run the start script with the --authn-cert flag and, optionally, the --auth-service-version flag.

The start script will download and set up the authenticator service from JFrog Artifactory, running it as a container in the same Docker Compose network as Conjur.

For the automatic download and setup to work, you need to have the JFrog CLI configured to point to the JFrog Artifactory instance where the authenticator service is published; see the start script for more details on the expected configuration.

If you need to use a custom or external authenticator service, you can skip the automatic setup by specifying local for the --auth-service-version flag and setting the relevant environment variables to point Conjur to the authenticator service.

Check ./start --help for more information on the usage of these flags.

Example usages:

# 1. Start the container using the latest version of the authenticator service downloaded from JFrog
./start --authn-cert

# 2. Start the container using a specific version (e.g., 333) of the authenticator service downloaded from JFrog
./start --authn-cert --auth-service-version 333

# 3. Skip the automatic download and authenticator service setup; Conjur will use a custom standalone service defined by environment variables
./start --authn-cert --auth-service-version local

For more information on how to set up and configure the authenticator service, please refer to AUTHENTICATOR_SERVICE.md

Visual Studio Code IDE Debugging

To debug Conjur using VS Code IDE, follow these steps:

  1. Ensure you have the "Ruby LSP" and "rdbg" extensions installed in VSCode

  2. Start or enter the development container.

  3. Instead of running conjurctl server, run the following command to start the server with the debugger:

    bundle exec rdbg --open --port=12345 --host=0.0.0.0 -c -- rails server -b 0.0.0.0 -u webrick

  4. Go to the "Run and Debug" tab in VS Code and click the "Start Debugging" button with the "Attach with rdbg (tcp 12345)" configuration selected. You should now be able to debug normally using breakpoints and the debug console.

Development CLI

The ./cli script in the dev folder provides a comprehensive set of commands for managing and testing your local Conjur development environment.

$ cd dev
$ ./cli --help

Server Management:

  • ./cli start - Start the Conjur development environment
  • ./cli stop - Stop all containers
  • ./cli restart - Restart the Conjur server
  • ./cli status - Check if the server is running
  • ./cli logs [service] - View container logs

Container Access:

  • ./cli exec - Open a bash shell in the Conjur container
  • ./cli key - Display the admin user's API key

Policy Management:

  • ./cli policy load <account> <policy-file> - Load a policy file

Testing Commands: The CLI includes built-in API testing commands that authenticate and make requests to your local Conjur instance:

  • ./cli test auth [user] - Authenticate as a user (admin or alice)
  • ./cli test whoami [user] - Get current user info
  • ./cli test roles - List all roles
  • ./cli test role [user] - Get role details
  • ./cli test policies - List policies
  • ./cli test cert-status - Check certificate authenticator status
  • ./cli test all - Run all test commands

Step into the running Conjur container

$ ./cli exec
root@88d43f7b3dfa:/src/conjur-server#

View the admin user's API key

$ ./cli key
3xmx4tn353q4m02f8e0xc1spj8zt6qpmwv178f5z83g6b101eepwn1

Load a policy

$ ./cli policy load cucumber policy/example.yml

Test API endpoints

$ ./cli test auth admin
$ ./cli test whoami admin

For most development work, the account will be cucumber, which is created when the development environment starts.

Updating the API

Are you planning a change to the Conjur API? This could involve adding a new endpoint, extending an existing endpoint, or changing the response of an existing endpoint. When you make changes to the Conjur API, you must also update the Conjur OpenAPI Spec.

To prepare to make a change to the Conjur API, follow the process below:

  1. Clone the OpenAPI spec project and create a branch.
  2. Update the spec with your planned API changes and create a draft pull request; make sure it references the Conjur issue you are working on. Note: it is expected that the automated tests in your spec branch will fail, because they are running against the conjur:edge image which hasn't been updated with your API changes yet.
  3. Return to your clone of the Conjur project, and make your planned changes to the Conjur API following the standard branch / review / merge workflow.
  4. Once your Conjur changes have been merged and the new conjur:edge image has been published, rerun the automation in your OpenAPI pull request to ensure that the spec is consistent with your API changes. Have your spec PR reviewed and merged as usual.

Note: Conjur's current API version is in the API_VERSION file and should correspond to the OpenAPI version.

Updating the database schema

The Conjur database schema is implemented as Sequel database migration files. To add a new database migration, run the command inside the Conjur development container:

$ rails generate migration <migration_name>
   ...
   create    db/migrate/20210315172159_migration_name.rb

This creates a new file under db/migrate with the migration name prefixed by a timestamp.

The initial contents of the file are similar to:

Sequel.migration do
  up do
    ...
  end

  down do
    ...
  end
end

More documentation on how to write Sequel migrations is available here.

Database migrations are applied automatically when starting Conjur with the conjurctl server command.

Testing

Conjur has rspec and cucumber tests, and an automated CI Pipeline.

Note on performance testing: set WEB_CONCURRENCY: 0 - this configuration is useful for recording accurate coverage data that can be used in theci/docker-compose.yml and conjur/ci/test_suites/authenticators_k8s/dev/dev_conjur.template.yaml. This isn't a realistic configuration and should not be used for benchmarking.

CI Pipeline

The CI Pipeline is defined in the Jenkinsfile, and documented in CI_README.md

RSpec

RSpec tests are easy to run from within the conjur server container:

root@aa8bc35ba7f4:/src/conjur-server# rspec
Run options: exclude {:performance=>true}

Randomized with seed 62317
.............................................

Finished in 3.84 seconds (files took 3.33 seconds to load)
2252 examples, 0 failures, 3 pending

Cucumber

Cucumber tests require the Conjur server to be running. It's easiest to achieve this by starting Conjur in one container and running Cucumber from another. Run the service in the conjur server container:

root@aa8bc35ba7f4:/src/conjur-server# conjurctl server
...
* Listening on tcp://localhost:3000
Use Ctrl-C to stop

Then, using the dev/cli script, step into the Conjur container to run the cukes:

$ ./cli exec
...
root@9feae5e5e001:/src/conjur-server#

Adding New Test Suites

When adding new test suites, please follow the guidelines in the top comments of the file ci/test.

Spin up Open ID Connect (OIDC) Compatible Environment for testing

To run the cukes with an Open ID Connect (OIDC) compatible environment, run cli with the --authn-oidc flag:

$ ./cli exec --authn-oidc
...
root@9feae5e5e001:/src/conjur-server#

Spin up Google Cloud Platform (GCP) Compatible Environment for testing

Prerequisites

To run the cukes with a Google Cloud Platform (GCP) compatible environment, run cli with the --authn-gcp flag and pass the following:

  1. The name of a running Google Compute Engine (GCE) instance. (for example: my-gce-instance)

  2. The URL of the Google Cloud Function (GCF). (for example: https://us-central1-exmaple.cloudfunctions.net/idtoken?audience=conjur/cucumber/host/demo-host)

$ ./cli exec --authn-gcp --gce [GCE_INSTANCE_NAME] --gcf [GCF_URL]
...
root@9feae5e5e001:/src/conjur-server#

When running with --authn-gcp flag, the cli script executes another script which does the heavy lifting of provisioning the ID tokens (required by the tests) from Google Cloud Platform. To run the GCP authenticator test suite:

root@9feae5e5e001:/src/conjur-server# cucumber -p authenticators_gcp cucumber/authenticators_gcp/features

Run all the cukes:

Below is the list of the available Cucumber suites:

  • api
  • authenticators_azure
  • authenticators_config
  • authenticators_gcp
  • authenticators_jwt
  • authenticators_ldap
  • authenticators_oidc
  • authenticators_cert
  • authenticators_status
  • manual-rotators
  • policy
  • rotators

Each of the above suites can be executed using a profile of the same name. For example, to execute the api suite, your command might look like the following:

root@9feae5e5e001:/src/conjur-server# cucumber --profile api  # runs api cukes

Run just one feature:

root@9feae5e5e001:/src/conjur-server# cucumber --profile api cucumber/api/features/resource_list.feature

Rake Tasks

Rake tasks are easy to run from within the conjur server container:

  • Get the next available error code from errors 
    root@aa8bc35ba7f4:/src/conjur-server# rake error_code:next
    The output will be similar to 
    The next available error number is 63 ( CONJ00063E )

Kubernetes specific Cucumber tests

Several cucumber tests are written to verify conjur works properly when authenticating to Kubernetes. These tests have hooks to run against both Openshift and Google GKE.

The cucumber tests are located under cucumber/authenticators_k8s/features and can be run by going into the ci/test_suites/authenticators_k8s directory and running:

$ summon -f [secrets.ocp.yml|secrets.yml] ./init_k8s.sh [openshift|gke]
$ summon -f [secrets.ocp.yml|secrets.yml] ./entrypoint.sh [openshift|gke]
  • init_k8s.sh - executes a simple login to Openshift or GKE to verify credentials as well as logging into the Docker Registry defined
  • test.sh - executes the tests against the defined platform

Secrets file

The secrets file used for summons needs to contain the following environment variables

  • openshift
    • OPENSHIFT_USERNAME - username of an account that can create namespaces, adjust cluster properties, etc
    • OPENSHIFT_PASSWORD - password of the account
    • OPENSHIFT_URL - the URL of the RedHat CRC cluster
      • If running this locally - use https://host.docker.internal:6443 so the docker container can talk to the CRC containers
    • OPENSHIFT_TOKEN - the login token of the above username/password
      • only needed for local execution because the docker container executing the commands can't redirect for login
      • obtained by running the following command locally after login - oc whoami -t
  • gke
    • GCLOUD_CLUSTER_NAME - cluster name of the GKE environment in the cloud
    • GCLOUD_ZONE - zone of the GKE environment in the cloud
    • GCLOUD_PROJECT_NAME - project name of the GKE environment
    • GCLOUD_SERVICE_KEY - service key of the GKE environment

Local Execution Prerequisites

To execute the tests locally, a few things will have to be done:

  • Openshift
    • Download and install the RedHat Code Ready Container
      • This contains all the necessary pieces to have a local version of Openshift
    • After install, copy down the kubeadmin username/password and update the secrets.ocp.yml file with the password
    • Execute oc whoami -t and update the token property
  • GKE
    • Work with infrastructure to obtain a GKE environment

If the local revision of your files don't have a docker image built yet - build the docker images using the following command:

$ ./build_locally.sh <sni cert file>

Pull Request Workflow

  1. Fork the project
  2. Clone your fork
  3. Make local changes to your fork by editing files
  4. Commit your changes
  5. Push your local changes to the remote server
  6. Create new Pull Request

From here your pull request will be reviewed and once you've responded to all feedback it will be merged into the project. Congratulations, you're a contributor!

Style guide

Use this guide to maintain consistent style across the Conjur project.

Changelog maintenance

The changelog file is maintained based on Keep a Changelog guidelines.

Each accepted change to the Conjur code (documentation and website updates excepted) requires adding a changelog entry to the corresponding Added, Changed, Deprecated, Removed, Fixed and/or Security sub-section (add one as necessary) of the Unreleased section in the changelog.

Bumping the version number after each and every change is not required, advised nor expected. Valid reasons to bump the version are for example:

  • Enough changes have accumulated,
  • An important feature has been implemented,
  • An external project depends on one of the recent changes.

Releasing

Verify and update dependencies

  1. Review the NOTICES.txt file and ensure it reflects the current set of dependencies in the Gemfile
  2. If a new dependency has been added, a dependency has been dropped, or a version has changed since the last tag - make sure the NOTICES file is up-to-date with the new versions

Update the version and changelog

  1. Examine the changelog and decide on the version bump rank (major, minor, patch).
  2. Change the title of Unreleased section of the changelog to the target version.
    • Be sure to add the date (ISO 8601 format) to the section header.
  3. Add a new, empty Unreleased section to the changelog.
    • Remember to update the references at the bottom of the document.
  4. Change VERSION file to reflect the change. This file is used by some scripts.
  5. Change the API_VERSION file to reflect the correct OpenAPI spec release if there has been an update to the API. If the OpenAPI spec is out of date with the current API, it will need to be updated and released before you can release this project.
  6. Create a branch and commit these changes (including the changes to NOTICES.txt, if there are any). Bump version to x.y.z is an acceptable commit message.
  7. Push your changes and get the PR reviewed and merged.

Tag the version

  1. Tag the version on the master branch using eg. git tag -s v1.2.3. Note this requires you to be able to sign releases. Consult the github documentation on signing commits on how to set this up.

    • Git will ask you to enter the tag message, which should just be v1.2.3.

  2. Push the tag: git push v1.2.3 (or git push origin v1.2.3 if you are working from your local machine).

Note: you may find it convenient to use the release script to add the tag. In general, deleting and changing tags should be avoided.

Add a new GitHub release

  1. Create a new release from the tag in the GitHub UI
  2. Add the CHANGELOG for the current version to the GitHub release description

Publishing images

Visit the Red Hat project page once the images have been pushed and manually choose to publish the latest release.