From fe689d4840381e58c83d10253ccd5045a4799eac Mon Sep 17 00:00:00 2001 From: Qubasa Date: Tue, 31 Oct 2023 12:57:28 +0100 Subject: [PATCH] Added contributing.md and improved README.md --- README.md | 26 ++++-- docs/contributing.md | 197 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 217 insertions(+), 6 deletions(-) create mode 100644 docs/contributing.md diff --git a/README.md b/README.md index e1f5ad04..6995b6ca 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,23 @@ -# clan.lol core +# cLAN Core Repository -This is the monorepo of the clan.lol project -In here are all the packages we use, all the nixosModules we use/expose, the CLI and tests for everything. +Welcome to the cLAN Core Repository, the heart of the [clan.lol](https://clan.lol/) project! This monorepo houses all the essential packages, NixOS modules, CLI tools, and tests you need to contribute and work with the cLAN project. -## cLAN config tool +## Getting Started -- The quickstart guide can be found here: [here](/clan/clan-core/src/branch/main/docs/quickstart.md) -- Find the docs [here](/clan/clan-core/src/branch/main/docs/clan-config.md) +If you're new to cLAN and eager to dive in, start with our quickstart guide: + +- **Quickstart Guide**: Check out [quickstart.md](docs/quickstart.md) to get up and running with cLAN in no time. + +## Managing Secrets + +Security is paramount, and cLAN provides guidelines for handling secrets effectively: + +- **Secrets Management**: Learn how to manage secrets securely by reading [secrets-management.md](docs/secrets-management.md). + +## Contributing to cLAN + +We welcome contributions from the community, and we've prepared a comprehensive guide to help you get started: + +- **Contribution Guidelines**: Find out how to contribute and make a meaningful impact on the cLAN project by reading [contributing.md](docs/contributing.md). + +Whether you're a newcomer or a seasoned developer, we look forward to your contributions and collaboration on the cLAN project. Let's build amazing things together! \ No newline at end of file diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 00000000..5a6f8358 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,197 @@ +# Website Template + +Welcome to our website template repository! This template is designed to help you and your team build high-quality websites efficiently. We've carefully chosen the technologies to make development smooth and enjoyable. Here's what you can expect from this template: + +**Frontend**: Our frontend is powered by [React NextJS](https://nextjs.org/), a popular and versatile framework for building web applications. + +**Backend**: For the backend, we use Python along with the [FastAPI framework](https://fastapi.tiangolo.com/). To ensure seamless communication between the frontend and backend, we generate an `openapi.json` file from the Python code, which defines the REST API. This file is then used with [Orval](https://orval.dev/) to generate TypeScript bindings for the REST API. We're committed to code correctness, so we use [mypy](https://mypy-lang.org/) to ensure that our Python code is statically typed correctly. For backend testing, we rely on [pytest](https://docs.pytest.org/en/7.4.x/). + +**Continuous Integration (CI)**: We've set up a CI bot that rigorously checks your code using the quality assurance (QA) tools mentioned above. If any errors are detected, it will block pull requests until they're resolved. + +**Dependency Management**: We use the [Nix package manager](https://nixos.org/) to manage dependencies and ensure reproducibility, making your development process more robust. + +# Getting Started with the Development Environment + +Let's get your development environment up and running: + +1. **Install Nix Package Manager**: + + - You can install the Nix package manager by either [downloading the Nix installer](https://github.com/DeterminateSystems/nix-installer/releases) or running this command: + ```bash + curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install + ``` + +2. **Install direnv**: + + - Download the direnv package from [here](https://direnv.net/docs/installation.html) or run the following command: + ```bash + curl -sfL https://direnv.net/install.sh | bash + ``` + +3. **Clone the Repository and Navigate**: + + - Clone this repository and navigate to it. + +4. **Allow .envrc**: + + - When you enter the directory, you'll receive an error message like this: + ```bash + direnv: error .envrc is blocked. Run `direnv allow` to approve its content + ``` + - Execute `direnv allow` to automatically execute the shell script `.envrc` when entering the directory. + +5. **Build the Backend**: + + - Go to the `pkgs/clan-cli` directory and execute: + ```bash + direnv allow + ``` + - Wait for the backend to build. + +6. **Start the Backend Server**: + + - To start the backend server, execute: + ```bash + clan webui --reload --no-open --log-level debug + ``` + - The server will automatically restart if any Python files change. + +7. **Build the Frontend**: + + - In a different shell, navigate to the `pkgs/ui` directory and execute: + ```bash + direnv allow + ``` + - Wait for the frontend to build. + +8. **Start the Frontend**: + - To start the frontend, execute: + ```bash + npm run dev + ``` + - Access the website by going to [http://localhost:3000](http://localhost:3000). + +# Setting Up Your Git Workflow + +Let's set up your Git workflow to collaborate effectively: + +1. **Register Your Gitea Account Locally**: + + - Execute the following command to add your Gitea account locally: + ```bash + tea login add + ``` + - Fill out the prompt as follows: + - URL of Gitea instance: `https://gitea.gchq.icu` + - Name of new Login [gitea.gchq.icu]: `gitea.gchq.icu:7171` + - Do you have an access token? No + - Username: YourUsername + - Password: YourPassword + - Set Optional settings: No + +2. **Git Workflow**: + + 1. Add your changes to Git using `git add `. + 2. Run `nix fmt` to lint your files. + 3. Commit your changes with a descriptive message: `git commit -a -m "My descriptive commit message"`. + 4. Make sure your branch has the latest changes from upstream by executing: + ```bash + git fetch && git rebase origin/main --autostash + ``` + 5. Use `git status` to check for merge conflicts. + 6. If conflicts exist, resolve them. Here's a tutorial for resolving conflicts in [VSCode](https://code.visualstudio.com/docs/sourcecontrol/overview#_merge-conflicts). + 7. After resolving conflicts, execute `git merge --continue` and repeat step 5 until there are no conflicts. + +3. **Create a Pull Request**: + + - To automatically open a pull request that gets merged if all tests pass, execute: + ```bash + merge-after-ci + ``` + +4. **Review Your Pull Request**: + + - Visit https://gitea.gchq.icu and go to the project page. Check under "Pull Requests" for any issues with your pull request. + +5. **Push Your Changes**: + - If there are issues, fix them and redo step 2. Afterward, execute: + ```bash + git push origin HEAD:YourUsername-main + ``` + - This will directly push to your open pull request. + +# Debugging + +When working on the backend of your project, debugging is an essential part of the development process. Here are some methods for debugging and testing the backend of your application: + +## Test Backend Locally in Devshell with Breakpoints + +To test the backend locally in a development environment and set breakpoints for debugging, follow these steps: + +1. Run the following command to execute your tests and allow for debugging with breakpoints: + ```bash + pytest -n0 -s --maxfail=1 + ``` + You can place `breakpoint()` in your Python code where you want to trigger a breakpoint for debugging. + +## Test Backend Locally in a Nix Sandbox + +To run your backend tests in a Nix sandbox, you have two options depending on whether your test functions have been marked as impure or not: + +### Running Tests Marked as Impure + +If your test functions need to execute `nix build` and have been marked as impure because you can't execute `nix build` inside a Nix sandbox, use the following command: + +```bash +nix run .#impure-checks +``` + +This command will run the impure test functions. + +### Running Pure Tests + +For test functions that have not been marked as impure and don't require executing `nix build`, you can use the following command: + +```bash +nix build .#checks.x86_64-linux.clan-pytest --rebuild +``` + +This command will run all pure test functions. + +### Inspecting the Nix Sandbox + +If you need to inspect the Nix sandbox while running tests, follow these steps: + +1. Insert an endless sleep into your test code where you want to pause the execution. For example: + + ```python + import time + time.sleep(3600) # Sleep for one hour + ``` + +2. Use `cntr` and `psgrep` to attach to the Nix sandbox. This allows you to interactively debug your code while it's paused. For example: + + ```bash + cntr exec -w your_sandbox_name + psgrep -a -x your_python_process_name + ``` + +These debugging and testing methods will help you identify and fix issues in your backend code efficiently, ensuring the reliability and robustness of your application. + +# Using this Template + +To make the most of this template: + +1. Set up a new Gitea account named `ui-asset-bot`. Generate an access token with all access permissions and set it under `settings/actions/secrets` as a secret called `BOT_ACCESS_TOKEN`. + + - Also, edit the file `.gitea/workflows/ui_assets.yaml` and change the `BOT_EMAIL` variable to match the email you set for that account. Gitea matches commits to accounts by their email address, so this step is essential. + +2. Create a second Gitea account named `merge-bot`. Edit the file `pkgs/merge-after-ci/default.nix` if the name should be different. Under "Branches," set the main branch to be protected and add `merge-bot` to the whitelisted users for pushing. Set the unprotected file pattern to `**/ui-assets.nix`. + + - Enable the status check for "build / test (pull_request)." + +3. Add both `merge-bot` and `ui-asset-bot` as collaborators. + - Set the option to "Delete pull request branch after merge by default." + - Also, set the default merge style to "Rebase then create merge commit." + +With this template, you're well-equipped to build and collaborate on high-quality websites efficiently. Happy coding!