docs: update documentation

Author: NejcKle

README.md

@@ -11,23 +11,50 @@ specific functionality. The main components include:
 - **Data Ingestion Service**: Responsible for collecting data from various sensor endpoints.
 - **Data Processing Service**: Processes the ingested data, applying any necessary transformations
   or aggregations.
-- **API Gateway**: Exposes a REST API for clients to access the processed data.
+- **REST API**: Exposes a REST API for clients to access the processed data.
+- **Application**: Sample frontend application to visualize and interact with the data.
+
+A technical overview of the architecture is shown below:
+
+![Architecture Diagram](./docs/architecture/architecture.png)
+
+_Components in grey are either optional or not implemented yet._
+
+Legend:
+
+- Purple blocks represent external services / devices.
+- Orange blocks represent services that are part of the system.
+- Blue blocks represent data storage and visualization components.
+- Green blocks represent user-facing applications.
+
+> [!NOTE] Kubernetes Autoscaling Kubernetes Autoscaling is an option, which can be implemented in
+> the future if the application scales up. This would allow the services to automatically adjust
+> their number of instances based on the current load and demand.
 
 ## Folder Structure
 
 ```text
 └── 📁ajdovscina-cloud-software
     └── 📁alertmanager
+    └── 📁application
+        └── 📁public
+            ├── favicon.ico
+            ├── index.html
+        └── 📁src
+            └── 📁components
+            └── 📁services
+            └── 📁types
+        ├── Makefile
     └── 📁common
         └── 📁logging
     └── 📁dockerfiles
     └── 📁docs
         └── 📁architecture
-            ├── ajdovscina-architecture.drawio
         └── 📁images
     └── 📁grafana
         └── 📁dashboards
     └── 📁ingest_service
+        └── 📁common
         └── 📁models
         └── 📁util
         ├── Makefile
@@ -36,11 +63,13 @@ specific functionality. The main components include:
     └── 📁mock
         └── 📁ingest_service
     └── 📁processor_service
+        └── 📁common
         └── 📁src
         └── 📁util
         ├── Makefile
     └── 📁prometheus
     └── 📁rest_api
+        └── 📁common
         └── 📁models
         └── 📁src
         └── 📁util
@@ -81,7 +110,61 @@ And to stop the production system:
 make prod-down
 ```
 
-> [!IMPORTANT] Each of the services can be deployed independently using their respective makefiles.
+> [!IMPORTANT] Service local deployment Each of the services can be deployed independently locally
+> using their respective makefiles. See the documentation in each service's directory for more
+> details. This should be used in conjunction with the `make dev-up` command to start the
+> dependencies.
+
+## Building Blocks
+
+The application is composed of several building blocks, each with its own purpose and functionality.
+These blocks are defined in the `docker-compose` files located in the root directory.
+
+These are subject to change as the system evolves, but currently include:
+
+- **PostgreSQL with TimescaleDB**: The primary database for storing sensor data and application
+  metadata.
+- **Kafka**: A message broker used for decoupling the ingestion and processing services.
+- **Prometheus**: A monitoring and alerting toolkit used for collecting and querying metrics.
+- **Grafana**: A visualization tool used for creating dashboards and visualizing metrics collected
+  by Prometheus.
+- **Alertmanager**: A component of the Prometheus ecosystem used for handling alerts sent by
+  Prometheus.
+- **NodeRED**: A flow-based development tool for visual programming, used for wiring together
+  hardware devices, APIs, and online services.
+- **Ingest Service**: A FastAPI service responsible for receiving and validating incoming sensor
+  data.
+- **Processor Service**: A service that processes the ingested data and stores it in the database.
+- **Frontend Application**: A React-based application for visualizing and interacting with the
+  sensor data.
+
+The [dockerfiles](./dockerfiles) directory contains Dockerfiles for custom services:
+
+- `Dockerfile.ingest_service`: Dockerfile for the Ingest Service.
+- `Dockerfile.processor_service`: Dockerfile for the Processor Service.
+- `Dockerfile.rest_api`: Dockerfile for the REST API.
+- `Dockerfile.application`: Dockerfile for the Frontend Application.
+
+## Service Accessibility
+
+The services are accessible via the following ports when deployed:
+
+| Service              | Local Port | External Port Production | Description                                |
+| -------------------- | ---------- | ------------------------ | ------------------------------------------ |
+| PostgreSQL           | 5432       | 13328                    | Database service                           |
+| Prometheus           | 9090       | 19290                    | Monitoring and alerting toolkit            |
+| Grafana              | 3000       | 13201                    | Visualization tool                         |
+| Alertmanager         | 9093       | 19193                    | Alert management                           |
+| NodeRED              | 1880       | 11880                    | Flow-based development tool                |
+| Ingest Service       | 5000       | 40005                    | Data ingestion endpoint                    |
+| REST API             | 5005       | 41005                    | REST API for accessing sensor data         |
+| Frontend Application | 3001       | 8004                     | User interface for visualizing sensor data |
+
+External ports are used to access the services from outside the Docker network, while local ports
+are used for communication between services within the Docker network.
+
+If the services are run using their respective makefiles, the ports may differ. See the Makefile in
+each service's directory for more details.
 
 ### Configuration and Constants
 
@@ -110,3 +193,46 @@ DB_CONFIG = {
 
 API_KEY = "your-secret-api-key"  # Replace with your actual key
 ```
+
+Detailed descriptions of each constant:
+
+- `LOG_LEVEL`: Sets the logging level for the application.
+- `DB_CONFIG`: A dictionary containing the database connection parameters:
+  - `dbname`: The name of the PostgreSQL database. Set to `ajdovscina` by default; This is defined
+    in the `docker-compose` files - `POSTGRES_DB` variable.
+  - `user`: The username used to connect to the database. Set to `ajdovscina-geospatial-user` by
+    default; This is defined in the `docker-compose` files - `POSTGRES_USER` variable.
+  - `password`: The password for the database user. Set to `password` by default; This is defined in
+    the `docker-compose` files - `POSTGRES_PASSWORD` variable.
+  - `host`: The hostname or IP address of the database server. Set to `localhost` for local
+    development or `geospatial-data` when using Docker Compose - that's the service name defined in
+    the `docker-compose` files.
+  - `port`: The port number on which the database server is listening. Set to `5432` for local
+    deployment or `13328` for Docker Compose - this is defined in the `docker-compose` files -
+    `ports` mapping in the `geospatial-data` service.
+
+#### ingest_service/constants.py
+
+The constants found in the ingest_service/constants.py file are specific to the Ingest Service.
+
+```python
+import os
+
+LOCALHOST = os.environ.get("LOCALHOST") == "true"
+
+KAFKA_HOST = "localhost:9992" if LOCALHOST else "kafka:9993"  # Kafka broker to connect to and produce messages to
+# kafka:9993 is the service name defined in the docker-compose files
+```
+
+#### processor_service/constants.py
+
+The constants found in the processor_service/constants.py file are specific to the Processor
+Service.
+
+```python
+import os
+
+LOCALHOST = os.environ.get("LOCALHOST") == "true"
+KAFKA_HOST = "localhost:9992" if LOCALHOST else "kafka:9993"  # Kafka broker to connect to and consume messages from
+# kafka:9993 is the service name defined in the docker-compose files
+```

application/README.md

@@ -1,55 +1,26 @@
-# Getting Started with Create React App
+# Sample Frontend Application
 
 This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
 
-## Available Scripts
+This simple frontend application allows users to visualize and interact with the sensor data
+ingested by the system. It fetches data from the REST API and displays it in a user-friendly manner.
 
-In the project directory, you can run:
+## Configuration
 
-### `npm start`
+A `.env` file is used to configure the application. The `.env` file should contain the following
+variables:
 
-Runs the app in the development mode.\
-Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
+```env
+PORT=3001
 
-The page will reload if you make edits.\
-You will also see any lint errors in the console.
+REACT_APP_API_TOKEN=your-secret-api-key
+REACT_APP_API_BASE_URL=http://localhost:51005
+```
 
-### `npm test`
+When deploying the application, make sure to set `REACT_APP_API_BASE_URL` to the URL where the REST
+API is hosted, and `REACT_APP_API_TOKEN` to a secure value that is set by the REST API.
 
-Launches the test runner in the interactive watch mode.\
-See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests)
-for more information.
+## Running the Application
 
-### `npm run build`
-
-Builds the app for production to the `build` folder.\
-It correctly bundles React in production mode and optimizes the build for the best performance.
-
-The build is minified and the filenames include the hashes.\
-Your app is ready to be deployed!
-
-See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for
-more information.
-
-### `npm run eject`
-
-**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
-
-If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time.
-This command will remove the single build dependency from your project.
-
-Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel,
-ESLint, etc) right into your project so you have full control over them. All of the commands except
-`eject` will still work, but they will point to the copied scripts so you can tweak them. At this
-point you’re on your own.
-
-You don’t have to ever use `eject`. The curated feature set is suitable for small and middle
-deployments, and you shouldn’t feel obligated to use this feature. However we understand that this
-tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
-
-## Learn More
-
-You can learn more in the
-[Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
-
-To learn React, check out the [React documentation](https://reactjs.org/).
+To run the application locally, run `make run-local-dev` from this directory. This will run the
+`npm run start` command and launch the application in development mode.

common/README.md

@@ -0,0 +1,12 @@
+# Common Utilities
+
+The common directory contains shared utilities and modules used across different services in the
+project. This includes database connection management, logging setup, and other helper functions.
+
+## Configuration
+
+The [constants.py](./constants.py) file contains configuration constants for the services. These
+should be configured based on the deployment environment.
+
+When deploying to the cloud, make sure you change the API key and database credentials to secure
+values.

common/constants.py

@@ -9,7 +9,7 @@
 DB_CONFIG = {
     "dbname": "ajdovscina",
     "user": "ajdovscina-geospatial-user",
-    "password": "password",
+    "password": "password",  # In production, use a secure password and manage it safely
     "host": "localhost" if localhost else "geospatial-data",
     "port": 13328 if localhost else 5432,
 }