Skip to content

Latest commit

 

History

History
629 lines (426 loc) · 22.7 KB

File metadata and controls

629 lines (426 loc) · 22.7 KB

Contributing to Nest

Thank you for considering contributing to the OWASP Nest project! This document provides guidelines to help you get started.

About the Project

Nest is a full-stack web application built using:

  • Backend: Python, Django
  • Frontend: TypeScript, Next.js, React, Tailwind CSS
  • Search: Algolia

The project uses a containerized approach for both development and production environments. Docker is required to run Nest locally.

Prerequisites

Before contributing, ensure you have the following installed:

  1. Docker for running the Nest containers.
  2. pre-commit for automated code checks.
  3. terraform and tflint for IaC.

Optional steps for Windows:

  1. WSL (Windows Subsystem for Linux) for Windows users to enable Linux compatibility
    1. The make run command requires WSL to function properly. Make sure WSL is installed and configured on your system.
    2. You must use WSL terminal (not Windows PowerShell) otherwise there is no guarantee that Nest development environment will be set up as intended. You can enter the Linux environment by running wsl. Please do not report any issues if you use PowerShell for running the commands -- it's not the intended way to run Nest locally so the errors will not be accepted as bugs.
    3. Ensure WSL integration is enabled in Docker Desktop settings by checking Resources -- WSL integration in Docker application settings.
    4. Cloning or running the project under /mnt/c (the Windows C: drive) can lead to significant performance degradation and Docker permission issues.

Starring the Project

GitHub stars

Forking the Repository

GitHub forks

External Documentation

Environment Variables

For detailed descriptions of all environment variables, see:

Setting up the Project

Follow these steps to set up the OWASP Nest application:

  1. Clone the Repository:

    • Clone the repository code from your GitHub account using the following command:

      git clone https://github.com/<your-account>/<nest-fork>
  2. Create Environment Files:

    • Copy the contents from the template file into your new backend local environment file:

      cp backend/.env.example backend/.env
    • Copy the contents from the template file into your new frontend local environment file:

      cp frontend/.env.example frontend/.env

Ensure that all .env files are saved in UTF-8 format without BOM (Byte Order Mark). This is crucial to prevent "Unexpected character" errors during application execution or Docker image building.

Please note you need to restart the application in order to apply any .env file changes.

  1. Configure Environment Variables:

    • Open the backend/.env file in your preferred text editor and change the DJANGO_CONFIGURATION value to Local:

      DJANGO_CONFIGURATION=Local
      
  2. Set Up Algolia:

    • Go to Algolia and create a free account.
    • An Algolia app is automatically created for you when you sign up.
    • During the sign up process, you may be asked to import data. You can skip this step.
    • Update your backend/.env file with the following keys from your Algolia app (use write API key for backend):
    DJANGO_ALGOLIA_APPLICATION_ID=<your-algolia-application-id>
    DJANGO_ALGOLIA_WRITE_API_KEY=<your-algolia-write-api-key>
    
    • Note: The default write API key should have index write permissions (addObject permission). If you do not use the default write API key, ensure that your API key has this permission.
    • If you encounter any issues, you can refer directly to Algolia's documentation
  3. Getting Started:

    • To see all available commands and their descriptions, run:

      make help
    • This will print a list of all make targets you can use during development.

  4. Run the Application:

    • In your terminal, navigate to the project root directory (not backend and not frontend subdirectories -- you need the project root directory) Nest has backend and frontend related Makefiles in corresponding directories and all of them are included in the main Makefile in the project root directory. Run the following command to start the application:

      make run
    • Leave this terminal session running and wait until you see that Nest local is responding.

    • Please note as we use containerized approach this command must be run in parallel to other Nest commands you may want to use. You need to keep it running in the current terminal and use another terminal session for your work.

  5. Load Initial Data:

    • Make sure you have gzip installed on your machine.

    • Open a new terminal session and run the following command to populate the database with initial data from fixtures:

    make load-data
  6. Index Data:

    • In the same terminal session, run the following command to index the data:

      make index-data
  7. Verify API Endpoints:

    • Check the following endpoints availability:

Optional Steps

GitHub OAuth Setup

If you want to use the "Sign in with GitHub" feature locally, you need to register a GitHub OAuth App and update your frontend/.env with real credentials. Without this, clicking "Sign in with GitHub" will result in a GitHub 404 error because the placeholder client_id=your-github-client-id is invalid.

  1. Go to GitHub Developer Settings → OAuth Apps and click "New OAuth App".

  2. Fill in the form with the following values:

    Field Value
    Application name OWASP Nest Local (or any name you prefer)
    Homepage URL http://localhost:3000
    Authorization callback URL http://localhost:3000/api/auth/callback/github
  3. Click "Register application".

  4. On the next page, copy the Client ID and click "Generate a new client secret" to obtain a Client Secret.

  5. Generate a secure NEXTAUTH_SECRET by running:

    openssl rand -base64 32
  6. Open your frontend/.env file and update the following values with the credentials you just obtained:

    NEXTAUTH_SECRET=<output-from-openssl-command>
    NEXT_SERVER_GITHUB_CLIENT_ID=<your-github-client-id>
    NEXT_SERVER_GITHUB_CLIENT_SECRET=<your-github-client-secret>
    
  7. Restart the application to apply the changes:

    make run

GitHub Data Fetch

If you plan to fetch GitHub OWASP data locally, follow these additional steps:

  1. Create a Super User:

    • Run the following command to create a super user for accessing the admin interface:

      make create-superuser
  2. Generate a GitHub Personal Access Token:

  3. Update Environment Variables with GitHub Token:

    • Open backend/.env again and update it with your GitHub token:

      GITHUB_TOKEN=<your-github-token>
      
  4. Sync Local Database Data:

    • Now you should be able to run the following command to sync your local database data with GitHub:

      make sync-data

NestBot Development

Never install your development Slack application in the OWASP Slack workspace.Doing so will interfere with OWASP Nest functionality and trigger unnecessary notifications to Slack admins.Always use a different workspace (create your own if needed).

To setup NestBot development environment, follow these steps:

  1. Set Up ngrok:

    • Go to ngrok and create a free account.

    • Install and configure ngrok on your machine using these instructions

    • Create your static domain by simply going to ngrok domains

    • Run the following commands to edit ngrok configuration:

      ngrok config edit
      agent:
          authtoken: <your-auth-token>
      tunnels:
          NestBot:
            addr: 8000
            proto: http
            hostname: <your-static-domain>
      
    • Now ngrok is all set, you access your local setup over internet, running the following command:

      ngrok start NestBot
  2. Update environment Variables with your NestBot Configuration:

    • Update backend/.env with your Slack application tokens:

      • Bot User OAuth Token from Settings -- Install App -- OAuth Tokens section
      • Signing Secret from Settings -- Basic Information -- App Credentials section
      DJANGO_SLACK_BOT_TOKEN=<your-slack-bot-token>
      DJANGO_SLACK_SIGNING_SECRET=<your-slack-signing-secret>
      
  3. Set up Slack application:

    • Configure your Slack application using NestBot manifest file (copy its contents and save it into Features -- App Manifest). You'll need to replace slash commands endpoint with your ngrok static domain path.
    • Reinstall your Slack application after making the changes using Settings -- Install App section.

Local Access to Internal Dashboards

When running the project locally, some UI sections are visible only if your user has the required backend roles configured via Django Admin. Follow the steps below to enable access during development.

Project Health Dashboard (Staff Access)

The Project Health Dashboard is visible only to users marked as staff.

Steps
  1. Start the backend server and open the Django Admin panel:

  2. Navigate to GitHub Users and open your user record.

  3. In the Permissions section, enable the custom is_owasp_staff field:

    • Locate the is_owasp_staff checkbox and tick it.
    • Save the changes.
  4. Clear authentication cookies:

    • Open http://localhost:3000
    • Open browser DevTools → Application / Storage
    • Clear Cookies for the site
    • Refresh the page and sign in again with GitHub.

After logging in, the Project Health Dashboard will appear in the user menu, or you can navigate directly to: http://localhost:3000/projects/dashboard

Mentorship Portal ("My Mentorship")

The "My Mentorship" menu item is shown only if the user is a Project Leader or a Mentor.

Option 1: Grant access as a Project Leader (recommended)
  1. Open Django Admin:
  2. Open an existing project or create a test project.
  3. In the leaders_raw field, add your exact GitHub username.
  4. Save the project.
Option 2: Grant access as a Mentor
  1. Ensure you have logged into the frontend at least once (so your GitHub user exists in the database).
  2. Open Django AdminMentorshipMentors.
  3. Click Add Mentor.
  4. Select your GitHub user in the github_user field.
  5. Fill in any required fields and save.
Refresh session

Role-based access is cached in the authentication cookie.

  1. Open http://localhost:3000
  2. Clear Cookies for the site
  3. Refresh and sign in again with GitHub

The "My Mentorship" option will now be visible in the user menu.

Code Quality Checks

Nest enforces code quality standards to ensure consistency and maintainability. You can run automated checks locally before pushing your changes:

make check

This command runs linters and other static analysis tools for both the frontend and backend.

We utilize third-party tools such as CodeRabbit, GitHub Advanced Security, and SonarQube for code review, static analysis, and quality checks. As a contributor, it's your responsibility to address (mark as resolved) all issues and suggestions reported by these tools during your pull request review. If a suggestion is valid, please implement it; if not, you may mark it as resolved with a brief explanation. If you're uncertain about a particular suggestion, feel free to leave a comment optionally tagging project maintainer(s) you're working with for further guidance.

Please note that your pull request will not be reviewed until all code quality checks pass and all automated suggestions have been addressed or resolved.

Testing

Our CI/CD pipelines automatically run tests against every Pull Request. You can run tests locally before submitting a PR:

make test

This command runs tests and checks that coverage threshold requirements are satisfied for both backend and frontend. Please note your PR won't be merged if it fails the code tests checks.

Running Security Scan

Run the security scans for vulnerabilities and anti-patterns with the following command:

make security-scan

This command automatically:

  • Performs local Semgrep and Trivy scans
  • Outputs findings to the terminal for immediate review

For addressing findings:

  • Review the output for specific file paths and line numbers
  • Follow the documentation links provided in the output for remediation guidance
  • Use # NOSEMGREP to suppress confirmed false positives while adding a short comment explaining each suppression

Running Code Scans Only

You can run code scan part separately via

make security-scan-code

Running Image Scans Only

You can run image scan part separately via

make security-scan-images

Running e2e Tests

Run the frontend e2e tests with the following command:

make test-frontend-e2e

This command automatically:

  • Starts the database and backend containers
  • Runs migrations and loads test data
  • Executes the e2e tests
  • Cleans up containers when done

For debugging, you can run the e2e backend separately:

make run-backend-e2e

Then load data manually in another terminal:

make load-data-e2e

Running Fuzz Tests

Run the fuzz tests with the following command:

make test-fuzz

This command automatically:

  • Starts the database and backend containers
  • Runs migrations and loads test data
  • Executes the fuzz tests
  • Cleans up containers when done

For debugging, you can run the fuzz backend separately:

make run-backend-fuzz

Then load data manually in another terminal:

make load-data-fuzz

Test Coverage

  • There is a minimum test coverage requirement for the backend code -- see pyproject.toml.
  • There is a minimum test coverage requirement for the frontend code -- see jest.config.ts.
  • Ensure your changes do not drop the overall test coverage percentage.

If you are adding new functionality, include relevant test cases.


Contributing Workflow

The following diagram illustrates the complete contribution workflow:

flowchart TD
    Start([Start]) --> CreateIssue[Create New Issue]
    Start --> FindIssue[Find Existing Issue]
    CreateIssue --> GetAssigned["**Get Assigned to Issue**<br/>PRs will be automatically<br/>closed if you're not assigned"]
    FindIssue --> GetAssigned
    GetAssigned --> ResolveIssue[**Resolve Issue**<br/>work on code/docs/tests updates]

    ResolveIssue --> RunChecks{**Run `make check-test`**<br/>locally! This is a required step -- you will not be assigned to new issues if you ignore this}
    RunChecks -->|Fails| WP1[ ]
    RunChecks -->|Passes| PushChanges[**Push Changes to<br/>GitHub Fork Branch**]
    WP1 -.-> ResolveIssue

    PushChanges --> HasPR{PR Exists?}
    HasPR -->|No| CreateDraftPR[Create Draft PR]
    HasPR -->|Yes| WaitAutoChecks[**Wait for Automated<br/>Checks to Finish**]
    CreateDraftPR --> WaitAutoChecks

    WaitAutoChecks --> CheckAutoTools{All **CodeRabbit and <br/>SonarQube** Comments<br/>Resolved?}
    CheckAutoTools -->|No| MarkDraft[Make Sure PR Is **Marked as a Draft**]
    CheckAutoTools -->|Yes| MarkReady[Mark PR as Ready<br/>for Review]
    MarkDraft --> WP2[ ]
    WP2 -.-> ResolveIssue

    MarkReady --> RequestReview[Request Review from<br/>Project Maintainers]
    RequestReview --> WaitMaintainer[Wait for Maintainers'<br/>Comments]

    WaitMaintainer --> HasMaintainerComments{**Maintainers' Comments<br/>Resolved**?}
    HasMaintainerComments -->|No| MarkDraft
    HasMaintainerComments -->|Yes| CheckCI{**CI/CD<br/>Passing?**}

    CheckCI -->|Yes| ReadyMerge([PR Ready for Merge])
    CheckCI -->|No| MarkDraft

    style Start fill:#4caf50,stroke:#2e7d32,stroke-width:2px,color:#ffffff
    style ReadyMerge fill:#4caf50,stroke:#2e7d32,stroke-width:2px,color:#ffffff
    style ResolveIssue fill:#ff9800,stroke:#f57c00,stroke-width:2px,color:#000000
    style RunChecks fill:#2196f3,stroke:#1565c0,stroke-width:2px,color:#ffffff
    style CheckAutoTools fill:#2196f3,stroke:#1565c0,stroke-width:2px,color:#ffffff
    style HasMaintainerComments fill:#2196f3,stroke:#1565c0,stroke-width:2px,color:#ffffff
    style CheckCI fill:#2196f3,stroke:#1565c0,stroke-width:2px,color:#ffffff
    style MarkDraft fill:#ff9800,stroke:#f57c00,stroke-width:2px,color:#000000
    style MarkReady fill:#4caf50,stroke:#2e7d32,stroke-width:2px,color:#ffffff
    style CreateDraftPR fill:#ff9800,stroke:#f57c00,stroke-width:2px,color:#000000
    style WP1 fill:transparent,stroke:transparent,color:transparent,width:0px,height:0px
    style WP2 fill:transparent,stroke:transparent,color:transparent,width:0px,height:0px

    linkStyle 0 stroke:#4caf50,stroke-width:2px
    linkStyle 1 stroke:#4caf50,stroke-width:2px
    linkStyle 2 stroke:#4caf50,stroke-width:2px
    linkStyle 3 stroke:#4caf50,stroke-width:2px
    linkStyle 4 stroke:#4caf50,stroke-width:2px
    linkStyle 5 stroke:#4caf50,stroke-width:2px
    linkStyle 6 stroke:#f44336,stroke-width:2px
    linkStyle 7 stroke:#4caf50,stroke-width:2px
    linkStyle 8 stroke:#f44336,stroke-width:2px
    linkStyle 9 stroke:#4caf50,stroke-width:2px
    linkStyle 10 stroke:#9e9e9e,stroke-width:2px
    linkStyle 11 stroke:#4caf50,stroke-width:2px
    linkStyle 12 stroke:#4caf50,stroke-width:2px
    linkStyle 13 stroke:#4caf50,stroke-width:2px
    linkStyle 14 stroke:#f44336,stroke-width:2px
    linkStyle 15 stroke:#4caf50,stroke-width:2px
    linkStyle 16 stroke:#f44336,stroke-width:2px
    linkStyle 17 stroke:#f44336,stroke-width:2px
    linkStyle 18 stroke:#4caf50,stroke-width:2px
    linkStyle 19 stroke:#4caf50,stroke-width:2px
    linkStyle 20 stroke:#4caf50,stroke-width:2px
    linkStyle 21 stroke:#f44336,stroke-width:2px
    linkStyle 22 stroke:#4caf50,stroke-width:2px
    linkStyle 23 stroke:#4caf50,stroke-width:2px
    linkStyle 24 stroke:#f44336,stroke-width:2px
Loading

Keep Your Fork in Sync with Upstream

To avoid working on an outdated copy of Nest (and to reduce merge conflicts), contributors may find it helpful to keep their fork synchronized with the main OWASP Nest repository.

Setting up the upstream remote

If you haven't added the upstream remote yet, add it using:

git remote add upstream https://github.com/OWASP/Nest.git

Verify that the upstream remote has been added by running:

git remote -v

This should show both origin (your fork) and upstream (the main repository) remotes.

Before working on a new feature or issue, update your local main branch from upstream/main:

git checkout main
git fetch upstream
git merge upstream/main

1. Find Something to Work On

  • Check the Issues tab for open issues: https://github.com/OWASP/Nest/issues
  • Found a bug or have a feature request? Open a new issue.
  • Want to work on an existing issue? Ask the maintainers to assign it to you before submitting a pull request.
  • New to the project? Start with issues labeled good first issue for an easier onboarding experience.

2. Create a Branch

Always create a feature branch for your work:

git checkout -b feature/my-feature-name

3. Make Changes and Commit

  • Check that your commits include only related and intended changes. Do not include unrelated files.

  • Follow best practices for code style and testing.

  • Add tests for any new functionality or changes to ensure proper coverage.

  • Run the code quality checks and tests:

    make check-test
  • Write meaningful commit messages:

    git commit -m "Add feature: short description"

4. Push Changes

  • Push your branch to the repository:

    git push origin feature/my-feature-name

5. Open a Pull Request

  • Submit a Pull Request (PR) to the main branch.
  • Your PR will trigger CI/CD pipelines that run automated checks and tests.

6. Review and Merge

  • Address feedback from maintainers during code review.
  • Once approved, your PR will be merged into the main branch.

Troubleshooting

  • "Unexpected character" error during application execution or Docker image building This error is usually caused by incorrect encoding of .env files.

    • Open the .env files in a text editor (e.g., VS Code) and save them as "UTF-8 without BOM":
      • Open the .env file in Visual Studio Code.
      • Click on the encoding information in the bottom-right corner of the window.
      • Select "Save with Encoding".
      • Choose "UTF-8" from the list (ensure it's not "UTF-8 with BOM").
    • Restart the application with make run and verify the error is resolved.

Code of Conduct

Please follow the Code of Conduct when interacting with other contributors.


Thank you for contributing to Nest! Your contributions help make this project better for everyone.