docs: project planning and architecture design
- Add ADR.md documenting all technology stack decisions with rationale - Create comprehensive ROADMAP.md with 10-phase implementation plan - Add 3 architecture diagrams (AWS infrastructure, application stack, network) - Document collaboration guidelines in `.github/copilot-instructions.md` - Technology stack descisions: AWS, Terraform, Ansible, Docker Compose, PostgreSQL Phase 1 establishes foundation for automated Gitea deployment with proper decision tracking and incremental development approach.
This commit is contained in:
aviyadeveloper
commit
ee5e319dad
84
.github/copilot-instructions.md
vendored
Normal file
84
.github/copilot-instructions.md
vendored
Normal file
@ -0,0 +1,84 @@
|
||||
# Copilot Instructions for Qvest Project
|
||||
|
||||
## Working Style: Pair Programming Mode
|
||||
|
||||
This project requires a **collaborative, incremental approach** where the user maintains 100% understanding and control.
|
||||
|
||||
### Core Principles
|
||||
|
||||
1. **Check In Frequently**
|
||||
- Before making any architectural or technical decision, present options and get approval
|
||||
- After completing a small unit of work, pause and check in
|
||||
- Never make implicit decisions by just writing them in documentation
|
||||
- User must approve every decision explicitly
|
||||
|
||||
2. **Work Incrementally**
|
||||
- Focus ONLY on the current phase/task
|
||||
- Do not create files, directories, or code for future phases
|
||||
- Each phase must be self-contained and commit-ready
|
||||
- Complete one thing fully before moving to the next
|
||||
|
||||
3. **Maintain Simplicity**
|
||||
- Prefer simple, clean, concise solutions
|
||||
- Avoid over-engineering and unnecessary complexity
|
||||
- Follow DRY (Don't Repeat Yourself) principles
|
||||
- Question complexity - is this really needed?
|
||||
|
||||
4. **Declarative Over Imperative**
|
||||
- Prefer declarative tools and approaches (Terraform, Docker Compose)
|
||||
- Avoid imperative scripts where declarative solutions exist
|
||||
- Configuration should be readable and maintainable
|
||||
|
||||
5. **Avoid Information Overload**
|
||||
- Do not create long markdown files in one go
|
||||
- Break documentation into small, digestible chunks
|
||||
- Present one section at a time and get feedback
|
||||
- Work slowly and methodically
|
||||
|
||||
### Workflow Pattern
|
||||
|
||||
For each task:
|
||||
|
||||
1. **Propose**: "Here's what I'm thinking for [X]. Does this approach work for you?"
|
||||
2. **Wait**: Get explicit approval before proceeding
|
||||
3. **Implement**: Make ONLY the changes discussed, nothing extra
|
||||
4. **Review**: Show what was done and confirm it meets expectations
|
||||
5. **Commit**: Ensure the work is ready to commit before moving on
|
||||
|
||||
### What NOT to Do
|
||||
|
||||
- ❌ Create multiple large files at once
|
||||
- ❌ Make decisions without explicit approval
|
||||
- ❌ Jump ahead to future phases
|
||||
- ❌ Create infrastructure for features not yet needed
|
||||
- ❌ Assume what the user wants
|
||||
- ❌ Rush through phases
|
||||
|
||||
### What TO Do
|
||||
|
||||
- ✅ Ask questions when uncertain
|
||||
- ✅ Present options with trade-offs
|
||||
- ✅ Work on one thing at a time
|
||||
- ✅ Keep changes small and focused
|
||||
- ✅ Explain rationale clearly but concisely
|
||||
- ✅ Ensure user understands every change
|
||||
|
||||
### Project-Specific Context
|
||||
|
||||
- **Goal**: Complete Platform Engineer interview task (Gitea deployment)
|
||||
- **Stack**: AWS, Terraform, Docker Compose, PostgreSQL
|
||||
- **Philosophy**: Simple, professional, production-grade but not over-engineered
|
||||
- **Learning Focus**: User must deeply understand all architecture and decisions
|
||||
- **Documentation**: ADR.md tracks all decisions with rationale and trade-offs
|
||||
|
||||
### Communication Style
|
||||
|
||||
- Be concise but clear
|
||||
- Ask before acting
|
||||
- Explain "why" not just "what"
|
||||
- Present trade-offs, not just recommendations
|
||||
- Pause frequently for feedback
|
||||
|
||||
---
|
||||
|
||||
**Remember**: This is pair programming. The user is the senior engineer making the decisions. Your role is to help implement their vision, not to run ahead with your own.
|
||||
234
ADR.md
Normal file
234
ADR.md
Normal file
@ -0,0 +1,234 @@
|
||||
# Architecture Decision Records (ADR)
|
||||
|
||||
This document tracks all significant architectural decisions made during the project, including rationale and trade-offs.
|
||||
|
||||
---
|
||||
|
||||
## ADR-001: Cloud Provider - AWS
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Use Amazon Web Services (AWS)
|
||||
|
||||
**Rationale**:
|
||||
- Industry-standard cloud provider with comprehensive service portfolio
|
||||
- Access to managed services when beneficial
|
||||
- Strong ecosystem and community support
|
||||
- Terraform has excellent AWS provider support
|
||||
|
||||
---
|
||||
|
||||
## ADR-002: Infrastructure as Code - Terraform
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Use Terraform for infrastructure provisioning
|
||||
|
||||
**Rationale**:
|
||||
- Declarative approach (aligns with project philosophy)
|
||||
- Industry standard for cloud infrastructure
|
||||
- Excellent AWS provider
|
||||
- State management enables reproducibility
|
||||
|
||||
**Scope**: VPC, EC2, Security Groups, S3, Route 53
|
||||
|
||||
---
|
||||
|
||||
## ADR-003: Configuration Management - Ansible
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Use Ansible for system configuration (kept minimal)
|
||||
|
||||
**Rationale**:
|
||||
- Avoids problematic user-data scripts (bad experience with debugging)
|
||||
- Idempotent - can re-run if setup fails
|
||||
- Real-time output visibility via SSH
|
||||
- Professional separation of concerns: Terraform (infra) → Ansible (config) → Docker (apps)
|
||||
|
||||
**Scope**: Install Docker, configure system basics, setup firewall
|
||||
**Philosophy**: Keep Ansible simple - no fancy roles or complexity
|
||||
|
||||
**Alternative Considered**: User-data scripts - rejected due to debugging difficulty and one-shot nature
|
||||
|
||||
---
|
||||
|
||||
## ADR-004: Application Deployment - Docker + Docker Compose
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Use Docker with Docker Compose for application orchestration
|
||||
|
||||
**Rationale**:
|
||||
- Fully declarative (docker-compose.yml)
|
||||
- Easy to test locally (dev/prod parity)
|
||||
- Simple version control and updates
|
||||
- Gitea has official Docker images
|
||||
- Portable and reproducible
|
||||
|
||||
**Scope**: Gitea, nginx, PostgreSQL, monitoring stack (later)
|
||||
|
||||
---
|
||||
|
||||
## ADR-005: Database - Self-Hosted PostgreSQL in Docker
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: PostgreSQL container, not RDS
|
||||
|
||||
**Rationale**:
|
||||
- Simpler architecture (everything in docker-compose.yml)
|
||||
- Shows ability to build and manage backups ourselves
|
||||
- More control over configuration
|
||||
- Cost-effective
|
||||
- PostgreSQL is Gitea's recommended database
|
||||
|
||||
**Trade-offs**:
|
||||
- **Pros**: Greater control, cost-effective, simpler architecture
|
||||
- **Cons**: Requires custom backup automation and testing
|
||||
|
||||
**Backup Strategy**: Custom scripts with pg_dump to S3 (detailed in backup phase)
|
||||
|
||||
**Future Consideration**: For higher availability requirements or larger scale, RDS would provide managed backups, point-in-time recovery, and Multi-AZ deployment
|
||||
|
||||
---
|
||||
|
||||
## ADR-006: Reverse Proxy - Nginx
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Nginx as reverse proxy
|
||||
|
||||
**Rationale**:
|
||||
- Lightweight and performant
|
||||
- Simple configuration for basic proxying
|
||||
- Industry standard
|
||||
- Works well in Docker
|
||||
|
||||
**Scope**: SSL termination, proxy to Gitea, HTTP→HTTPS redirect
|
||||
|
||||
---
|
||||
|
||||
## ADR-007: SSL Certificates - Let's Encrypt
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Let's Encrypt with certbot
|
||||
|
||||
**Rationale**:
|
||||
- Free, automated, trusted certificates
|
||||
- Widely accepted by all browsers (no certificate warnings)
|
||||
- Auto-renewal reduces operational burden
|
||||
- Industry-standard solution for SSL/TLS
|
||||
|
||||
**Requirement**: Valid domain name pointing to server
|
||||
|
||||
---
|
||||
|
||||
## ADR-008: Update Automation - Watchtower
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted
|
||||
|
||||
**Decision**: Watchtower for Docker image updates
|
||||
|
||||
**Rationale**:
|
||||
- Purpose-built for Docker environments
|
||||
- Simple to configure (runs as container)
|
||||
- Automatic image updates on schedule
|
||||
- Minimal complexity
|
||||
|
||||
**Scope**: Monitor and update Gitea, nginx, and other containers
|
||||
|
||||
---
|
||||
|
||||
## ADR-009: Monitoring - Prometheus + Grafana
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted (implementation later)
|
||||
|
||||
**Decision**: Prometheus for metrics, Grafana for visualization
|
||||
|
||||
**Rationale**:
|
||||
- Industry standard monitoring stack
|
||||
- Powerful querying with PromQL
|
||||
- Rich visualization and alerting capabilities
|
||||
- Strong community and pre-built dashboards
|
||||
|
||||
**Note**: To be implemented in later phase
|
||||
|
||||
---
|
||||
|
||||
## ADR-010: Logging - Loki + Promtail
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted (implementation later)
|
||||
|
||||
**Decision**: Loki for log aggregation, Promtail for collection
|
||||
|
||||
**Rationale**:
|
||||
- Lightweight compared to ELK stack
|
||||
- Integrates with Grafana (single pane of glass)
|
||||
- Good fit for Docker environments
|
||||
|
||||
**Note**: To be implemented in later phase
|
||||
|
||||
---
|
||||
|
||||
## ADR-011: Backup Strategy - Custom Scripts + S3
|
||||
|
||||
**Date**: 2026-06-08
|
||||
**Status**: Accepted (implementation later)
|
||||
|
||||
**Decision**: Bash scripts with pg_dump and AWS S3
|
||||
|
||||
**Rationale**:
|
||||
- Simple and maintainable
|
||||
- Full control over backup process and scheduling
|
||||
- S3 provides highly durable storage (99.999999999%)
|
||||
- Easy to test and validate restore procedures
|
||||
|
||||
**Scope**:
|
||||
- Database backups (pg_dump)
|
||||
- Gitea repository data
|
||||
- Configuration files
|
||||
- Automated scheduling with cron
|
||||
|
||||
**Note**: Details to be designed in backup phase
|
||||
|
||||
---
|
||||
|
||||
## Technology Stack Summary
|
||||
|
||||
| Layer | Technology | Rationale |
|
||||
|-------|-----------|-----------|
|
||||
| **Cloud** | AWS | Industry standard |
|
||||
| **Infrastructure** | Terraform | Declarative IaC |
|
||||
| **Configuration** | Ansible (minimal) | System setup, avoids user-data |
|
||||
| **Compute** | EC2 | Flexible VM hosting |
|
||||
| **Application** | Docker Compose | Declarative orchestration |
|
||||
| **Database** | PostgreSQL (Docker) | Self-managed, shows control |
|
||||
| **Reverse Proxy** | Nginx | Lightweight, standard |
|
||||
| **SSL** | Let's Encrypt | Free, automated, professional |
|
||||
| **DNS** | Route 53 | AWS-native |
|
||||
| **Updates** | Watchtower | Docker-native automation |
|
||||
| **Backups** | Scripts + S3 | Custom, controlled |
|
||||
| **Monitoring** | Prometheus + Grafana | Industry standard |
|
||||
| **Logging** | Loki + Promtail | Lightweight, integrated |
|
||||
|
||||
---
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **Simplicity First**: Avoid overengineering
|
||||
2. **Declarative Over Imperative**: Terraform, Docker Compose
|
||||
3. **Infrastructure as Code**: Everything version-controlled
|
||||
4. **Show Control**: Build things ourselves where it demonstrates skill
|
||||
5. **Professional**: Production-grade practices
|
||||
313
ROADMAP.md
Normal file
313
ROADMAP.md
Normal file
@ -0,0 +1,313 @@
|
||||
# Roadmap
|
||||
|
||||
This is the implementation road map for the project. It outlines the key milestones and features in incremental steps, allowing for a structured approach to development and deployment.
|
||||
|
||||
## Phase 1: Conceptualization and Planning
|
||||
|
||||
This phase will be achieved through discussion and research and will include the following steps (no code should be implemented in this phase):
|
||||
|
||||
### 1.1 Requirements Analysis
|
||||
- Define the scope and requirements of the project
|
||||
- Identify constraints and non-functional requirements
|
||||
- Determine host environment (cloud provider, VPS, or local)
|
||||
|
||||
### 1.2 Technology Selection ✅
|
||||
**Decisions documented in [ADR.md](ADR.md)**
|
||||
|
||||
- **Cloud**: AWS
|
||||
- **Infrastructure as Code**: Terraform
|
||||
- **Configuration Management**: Ansible (kept minimal)
|
||||
- **Application Deployment**: Docker + Docker Compose
|
||||
- **Database**: PostgreSQL (self-hosted in Docker)
|
||||
- **Reverse Proxy**: Nginx
|
||||
- **SSL**: Let's Encrypt with certbot
|
||||
- **Update Automation**: Watchtower
|
||||
- **Monitoring**: Prometheus + Grafana (later phase)
|
||||
- **Logging**: Loki + Promtail (later phase)
|
||||
- **Backup**: Custom scripts + S3 (later phase)
|
||||
|
||||
### 1.3 Architecture Design ✅
|
||||
- ✅ Overall system architecture designed
|
||||
- ✅ Network topology planned (VPC, subnets, security groups)
|
||||
- ✅ Three architecture diagrams created in docs/diagrams/
|
||||
|
||||
### 1.4 Project Structure ✅
|
||||
- Directory structure planned (will create incrementally per phase)
|
||||
- Documentation structure in place (`docs/diagrams/`)
|
||||
- Naming conventions: lowercase, hyphens for files, descriptive names
|
||||
|
||||
### Goals:
|
||||
- ✅ A clear full Roadmap for the project available in this file
|
||||
- ✅ Technology stack documented with rationale (see ADR.md)
|
||||
- ✅ Architecture diagrams created (3 diagrams in docs/diagrams/)
|
||||
- ✅ Project structure planned
|
||||
|
||||
**Phase 1 Complete!** Ready to begin Phase 2 (Infrastructure Setup).
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Infrastructure Setup
|
||||
|
||||
This phase focuses on preparing the hosting environment and basic infrastructure components.
|
||||
|
||||
### 2.1 AWS Account Setup
|
||||
- Configure AWS CLI and credentials
|
||||
- Set up Terraform backend (S3 + DynamoDB for state locking)
|
||||
- Create Route 53 hosted zone for domain
|
||||
- Set up budget alerts and cost monitoring
|
||||
|
||||
### 2.2 Terraform Infrastructure Foundation
|
||||
- Create VPC with public/private subnets
|
||||
- Set up Internet Gateway and NAT Gateway (if needed)
|
||||
- Configure Security Groups (EC2, RDS, ALB rules)
|
||||
- Provision EC2 instance with appropriate IAM role
|
||||
- Set up RDS PostgreSQL instance
|
||||
- Configure Route 53 DNS records
|
||||
- Install Docker and Docker Compose on EC2 (user data script)
|
||||
|
||||
### 2.3 Security Hardening
|
||||
- Configure SSH key-based authentication only
|
||||
- Set up bastion host or Systems Manager Session Manager
|
||||
- Enable VPC Flow Logs
|
||||
- Configure CloudTrail for audit logging
|
||||
- Apply least-privilege IAM policies
|
||||
|
||||
### Goals:
|
||||
- AWS infrastructure provisioned via Terraform
|
||||
- EC2 instance running and accessible
|
||||
- RDS PostgreSQL ready
|
||||
- Domain DNS configured
|
||||
- Security hardening complete
|
||||
- Infrastructure can be destroyed and recreated from code
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Automated Gitea Deployment
|
||||
|
||||
This phase implements the automated, reproducible Gitea installation.
|
||||
|
||||
### 3.1 Database Setup
|
||||
- Automate database installation (PostgreSQL/MariaDB/MySQL)
|
||||
- Create Gitea database and user
|
||||
- Configure database for production use
|
||||
- Secure database access
|
||||
|
||||
### 3.2 Gitea Installation
|
||||
- Create automation scripts/playbooks for Gitea installation
|
||||
- Configure Gitea application settings
|
||||
- Set up file storage and data directories
|
||||
- Configure Gitea to use database
|
||||
|
||||
### 3.3 Reverse Proxy Configuration
|
||||
- Install and configure reverse proxy (nginx/Apache)
|
||||
- Generate/configure SSL certificates
|
||||
- Configure proxy to forward to Gitea
|
||||
- Ensure Gitea UI is only accessible via proxy
|
||||
- Set up HTTP to HTTPS redirect
|
||||
|
||||
### 3.4 Testing
|
||||
- Test Gitea accessibility via HTTPS
|
||||
- Verify direct access to Gitea is blocked
|
||||
- Test Gitea functionality (create user, repo, etc.)
|
||||
- Validate automation by destroying and recreating environment
|
||||
|
||||
### Goals:
|
||||
- Gitea running and accessible via HTTPS through reverse proxy
|
||||
- Installation fully automated and reproducible
|
||||
- Documentation of deployment process
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Update Automation
|
||||
|
||||
This phase implements automated update mechanisms for Gitea and related components.
|
||||
|
||||
### 4.1 Update Strategy Design
|
||||
- Define update schedule (when to check/apply updates)
|
||||
- Define rollback strategy
|
||||
- Plan pre-update backup automation
|
||||
|
||||
### 4.2 Update Automation Implementation
|
||||
- Implement automated update mechanism
|
||||
- Configure pre-update health checks
|
||||
- Configure post-update validation
|
||||
- Set up update notifications
|
||||
- Test update process
|
||||
|
||||
### Goals:
|
||||
- Automated update system operational
|
||||
- Update process tested and validated
|
||||
- Rollback procedure documented
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Backup Strategy Implementation
|
||||
|
||||
This phase implements comprehensive backup solutions.
|
||||
|
||||
### 5.1 Backup Concept Document
|
||||
- Document backup strategy (3-2-1 rule)
|
||||
- Define backup scope (database, repos, config, etc.)
|
||||
- Define retention policy
|
||||
- Define RTO and RPO targets
|
||||
|
||||
### 5.2 Backup Implementation
|
||||
- Automate database backups
|
||||
- Automate Gitea data directory backups
|
||||
- Automate configuration backups
|
||||
- Set up backup storage (local + remote)
|
||||
- Implement backup rotation and cleanup
|
||||
- Schedule automated backups
|
||||
|
||||
### 5.3 Recovery Testing
|
||||
- Document restore procedures
|
||||
- Test database restore
|
||||
- Test full system restore
|
||||
- Document recovery time
|
||||
|
||||
### Goals:
|
||||
- Automated backup system operational
|
||||
- Restore procedures tested and documented
|
||||
- Backup strategy document completed
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Monitoring Implementation
|
||||
|
||||
This phase implements monitoring for system health and performance.
|
||||
|
||||
### 6.1 Monitoring Concept Document
|
||||
- Define key metrics to monitor
|
||||
- Define alerting thresholds
|
||||
- Define alert channels (email, Slack, etc.)
|
||||
|
||||
### 6.2 Monitoring Setup
|
||||
- Deploy monitoring solution
|
||||
- Configure system metrics collection (CPU, RAM, disk, network)
|
||||
- Configure Gitea-specific metrics
|
||||
- Configure database metrics
|
||||
- Set up monitoring dashboards
|
||||
- Configure alerting rules
|
||||
|
||||
### 6.3 Testing
|
||||
- Simulate failure scenarios
|
||||
- Verify alerts trigger correctly
|
||||
- Validate dashboard accuracy
|
||||
|
||||
### Goals:
|
||||
- Monitoring system operational with dashboards
|
||||
- Alerting configured and tested
|
||||
- Monitoring concept document completed
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Logging Implementation
|
||||
|
||||
This phase implements centralized logging for all components.
|
||||
|
||||
### 7.1 Logging Concept Document
|
||||
- Define logging architecture
|
||||
- Define log retention policy
|
||||
- Define log analysis requirements
|
||||
|
||||
### 7.2 Logging Setup
|
||||
- Deploy centralized logging solution
|
||||
- Configure Gitea application logging
|
||||
- Configure reverse proxy access logs
|
||||
- Configure database logs
|
||||
- Configure system logs collection
|
||||
- Set up log parsing and indexing
|
||||
- Create log search and visualization dashboards
|
||||
|
||||
### 7.3 Testing
|
||||
- Verify logs are being collected
|
||||
- Test log search functionality
|
||||
- Test log-based alerts (if applicable)
|
||||
|
||||
### Goals:
|
||||
- Centralized logging operational
|
||||
- All components sending logs to central system
|
||||
- Logging concept document completed
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Redundancy and High Availability
|
||||
|
||||
This phase implements fail-safe operations and redundancy.
|
||||
|
||||
### 8.1 Redundancy Concept Document
|
||||
- Document SPOF (Single Points of Failure) analysis
|
||||
- Design HA architecture
|
||||
- Define failover strategy
|
||||
- Define acceptable downtime
|
||||
|
||||
### 8.2 Redundancy Implementation (Optional/Simplified)
|
||||
- Implement database redundancy (replication/clustering) OR document approach
|
||||
- Implement application redundancy (multiple Gitea instances) OR document approach
|
||||
- Implement load balancing OR document approach
|
||||
- Document manual failover procedures
|
||||
|
||||
### Goals:
|
||||
- Redundancy concept document completed
|
||||
- PoC or detailed plan for HA implementation
|
||||
- Failover procedures documented
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Documentation and Final Testing
|
||||
|
||||
This phase consolidates all documentation and performs end-to-end testing.
|
||||
|
||||
### 9.1 Documentation
|
||||
- Create comprehensive README
|
||||
- Document architecture with diagrams
|
||||
- Document all procedures (deployment, updates, backup/restore, failover)
|
||||
- Create runbooks for common scenarios
|
||||
- Document interview discussion points
|
||||
|
||||
### 9.2 Final Testing
|
||||
- Perform end-to-end deployment test
|
||||
- Test all automated processes
|
||||
- Verify all documentation is accurate
|
||||
- Test system under load (optional)
|
||||
|
||||
### 9.3 Repository Organization
|
||||
- Store all code and docs in Gitea repository
|
||||
- Ensure repository is well-organized
|
||||
- Add proper README and documentation
|
||||
|
||||
### Goals:
|
||||
- Complete documentation package
|
||||
- All automation tested and validated
|
||||
- Ready for interview presentation
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Interview Preparation
|
||||
|
||||
This phase prepares for the interview discussion.
|
||||
|
||||
### 10.1 Preparation
|
||||
- Review all concept documents
|
||||
- Prepare to explain technology choices
|
||||
- Prepare architecture diagrams for presentation
|
||||
- Prepare to demonstrate the system
|
||||
- List lessons learned and trade-offs made
|
||||
- Prepare improvement suggestions
|
||||
|
||||
### Goals:
|
||||
- Ready to discuss all aspects of the implementation
|
||||
- Demo environment functional and accessible
|
||||
- Confident in technology choices and concepts
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ✅ Gitea accessible via HTTPS through reverse proxy
|
||||
- ✅ Installation fully automated and reproducible
|
||||
- ✅ Automated updates configured and tested
|
||||
- ✅ Comprehensive concept documents for: Backup, Monitoring, Logging, Redundancy
|
||||
- ✅ At least one PoC implementation (optional but recommended)
|
||||
- ✅ All code and documentation in Git repository
|
||||
- ✅ System accessible to interviewer over internet
|
||||
40
TASK.md
Normal file
40
TASK.md
Normal file
@ -0,0 +1,40 @@
|
||||
# DevOps Challenge: Microservice with Update Automation
|
||||
|
||||
The DevOps Challenge consists of two parts:
|
||||
|
||||
1. (Deployment)[#deployment]: Installation of an application with automated updates
|
||||
2. (Concept)[#concept]: Strategy for backup, redundancy, monitoring, and logging
|
||||
|
||||
The scripts, data, concepts, etc., created for these tasks should be stored in a Git repository within the Gitea installation.
|
||||
|
||||
Note: The use of AI tools such as ChatGPT or GitHub Copilot is explicitly
|
||||
permitted.
|
||||
|
||||
## Deployment
|
||||
|
||||
### Installation
|
||||
Your team has decided to use the DevOps platform Gitea and wants to run its own instance. Perform the deployment on a host system (cloud subscription, VPS, or your own laptop) that is accessible from the internet by your interviewer, while adhering to the following requirements:
|
||||
|
||||
- Choice of installation source: source code, package manager, or Docker
|
||||
- Installation must be automated and reproducible (free choice of configuration management system)
|
||||
- Use of a reverse proxy and SSL (self-signed is sufficient)
|
||||
- UI must be accessible only via the proxy
|
||||
- Setup and integration of a database (PostgreSQL, MariaDB, or MySQL)
|
||||
|
||||
### Update Automation
|
||||
Once Gitea is successfully set up, configure automation for the update process
|
||||
using a tool of your choice.
|
||||
|
||||
## Concept
|
||||
|
||||
For Backup, Redundancy, Monitoring, and Logging develop and document your concept regarding the following topics that will be discussed during the interview:
|
||||
|
||||
- Backup
|
||||
- Centralized logging of components
|
||||
- Monitoring of key functions
|
||||
- Fail-safe operation of the instance
|
||||
|
||||
### Optional
|
||||
|
||||
Creation of a proof-of-concept for one or more aspects of your concept
|
||||
(backup, monitoring, logging, redundancy)
|
||||
57
docs/diagrams/application-stack.md
Normal file
57
docs/diagrams/application-stack.md
Normal file
@ -0,0 +1,57 @@
|
||||
# Application Stack Diagram
|
||||
|
||||
This diagram shows the Docker containers running on the EC2 instance and their relationships.
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%
|
||||
|
||||
graph TB
|
||||
User([User])
|
||||
LetsEncrypt[Let's Encrypt<br/>SSL Certs]
|
||||
|
||||
subgraph EC2["EC2 Instance"]
|
||||
subgraph Docker["Docker Compose"]
|
||||
Nginx[Nginx<br/>Port 80, 443]
|
||||
Gitea[Gitea<br/>Port 3000]
|
||||
Postgres[(PostgreSQL<br/>Port 5432)]
|
||||
Watchtower[Watchtower<br/>Auto-updater]
|
||||
|
||||
Nginx -->|Reverse Proxy| Gitea
|
||||
Gitea -->|Database Connection| Postgres
|
||||
Watchtower -.->|Monitors & Updates| Nginx
|
||||
Watchtower -.->|Monitors & Updates| Gitea
|
||||
end
|
||||
end
|
||||
|
||||
User -->|HTTPS| Nginx
|
||||
LetsEncrypt -.->|Certbot Renewal| Nginx
|
||||
|
||||
style EC2 fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
|
||||
style Docker fill:#d1d5db,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
|
||||
|
||||
style Nginx fill:#10B981,stroke:#333,stroke-width:1px,color:#fff
|
||||
style Gitea fill:#3B82F6,stroke:#333,stroke-width:1px,color:#fff
|
||||
style Postgres fill:#8B5CF6,stroke:#333,stroke-width:1px,color:#fff
|
||||
style Watchtower fill:#F59E0B,stroke:#333,stroke-width:1px,color:#fff
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
- **Nginx**: Reverse proxy handling SSL termination and routing to Gitea
|
||||
- **Gitea**: Git server application (main service)
|
||||
- **PostgreSQL**: Database storing repositories metadata, users, issues
|
||||
- **Watchtower**: Monitors Docker Hub for image updates, automatically pulls and restarts containers
|
||||
|
||||
## Container Communication
|
||||
|
||||
- All containers in the same Docker network
|
||||
- Nginx proxies HTTPS requests to Gitea's internal port 3000
|
||||
- Gitea connects to PostgreSQL via container name
|
||||
- Watchtower runs on schedule, checking for updates
|
||||
- Let's Encrypt certbot renews certificates automatically (via nginx container or separate container)
|
||||
|
||||
## Data Persistence
|
||||
|
||||
Docker volumes ensure data survives container restarts:
|
||||
- `gitea_data`: Git repositories and uploads
|
||||
- `postgres_data`: Database files
|
||||
46
docs/diagrams/aws-infrastructure.md
Normal file
46
docs/diagrams/aws-infrastructure.md
Normal file
@ -0,0 +1,46 @@
|
||||
# AWS Infrastructure Diagram
|
||||
|
||||
This diagram shows the high-level AWS resources and their relationships.
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%
|
||||
|
||||
graph TB
|
||||
Internet([Internet/Users])
|
||||
Route53[Route 53<br/>DNS]
|
||||
EC2[EC2 Instance<br/>Docker Host]
|
||||
S3[(S3 Bucket<br/>Backups)]
|
||||
|
||||
Internet -->|HTTPS| Route53
|
||||
Route53 -->|DNS Resolution| EC2
|
||||
EC2 -->|Backup Upload| S3
|
||||
|
||||
subgraph AWS["AWS Account"]
|
||||
subgraph VPC["VPC"]
|
||||
EC2
|
||||
end
|
||||
Route53
|
||||
S3
|
||||
end
|
||||
|
||||
style AWS fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
|
||||
style VPC fill:#d1d5db,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
|
||||
|
||||
style EC2 fill:#10B981,stroke:#333,stroke-width:1px,color:#fff
|
||||
style S3 fill:#F97316,stroke:#333,stroke-width:1px,color:#fff
|
||||
style Route53 fill:#6366F1,stroke:#333,stroke-width:1px,color:#fff
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
- **Route 53**: DNS service that points domain to EC2 instance
|
||||
- **EC2 Instance**: Single VM running Docker with all application containers
|
||||
- **S3 Bucket**: Storage for database and application backups
|
||||
- **VPC**: Isolated network containing EC2 instance
|
||||
|
||||
## Traffic Flow
|
||||
|
||||
1. User accesses `gitea.yourdomain.com`
|
||||
2. Route 53 resolves to EC2 public IP
|
||||
3. Request hits EC2 (nginx handles SSL, proxies to Gitea)
|
||||
4. EC2 regularly backs up data to S3
|
||||
78
docs/diagrams/network-architecture.md
Normal file
78
docs/diagrams/network-architecture.md
Normal file
@ -0,0 +1,78 @@
|
||||
# Network Architecture Diagram
|
||||
|
||||
This diagram shows the VPC network design, subnets, and security group rules.
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%
|
||||
|
||||
graph TB
|
||||
Internet([Internet])
|
||||
IGW[Internet Gateway]
|
||||
|
||||
subgraph VPC["VPC 10.0.0.0/16"]
|
||||
IGW
|
||||
|
||||
subgraph PublicSubnet["Public Subnet 10.0.1.0/24"]
|
||||
EC2[EC2 Instance<br/>Docker Host]
|
||||
end
|
||||
|
||||
subgraph SG["Security Group: EC2"]
|
||||
direction TB
|
||||
SGRules["Inbound:<br/>- 22 SSH from Admin IP<br/>- 80 HTTP from 0.0.0.0/0<br/>- 443 HTTPS from 0.0.0.0/0<br/><br/>Outbound:<br/>- All traffic"]
|
||||
end
|
||||
end
|
||||
|
||||
Internet -->|HTTPS/HTTP| IGW
|
||||
IGW --> PublicSubnet
|
||||
EC2 -.->|Protected by| SG
|
||||
|
||||
style VPC fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
|
||||
style PublicSubnet fill:#d1d5db,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
|
||||
style SG fill:#f3f4f6,stroke:#6b7280,stroke-width:1px,stroke-dasharray: 5 5
|
||||
|
||||
style EC2 fill:#10B981,stroke:#333,stroke-width:1px,color:#fff
|
||||
style IGW fill:#6366F1,stroke:#333,stroke-width:1px,color:#fff
|
||||
style SGRules fill:#EF4444,stroke:#333,stroke-width:1px,color:#fff
|
||||
```
|
||||
|
||||
## Network Components
|
||||
|
||||
### VPC Configuration
|
||||
- **CIDR Block**: `10.0.0.0/16` (65,536 IP addresses)
|
||||
- **Region**: Single AWS region
|
||||
- **DNS Hostnames**: Enabled
|
||||
- **DNS Resolution**: Enabled
|
||||
|
||||
### Subnets
|
||||
- **Public Subnet**: `10.0.1.0/24` (256 IP addresses)
|
||||
- Auto-assign public IP: Enabled
|
||||
- Contains: EC2 instance
|
||||
- Route table: Routes to Internet Gateway
|
||||
|
||||
### Internet Gateway
|
||||
- Attached to VPC
|
||||
- Enables internet access for resources in public subnet
|
||||
|
||||
### Security Groups
|
||||
|
||||
**EC2 Security Group**:
|
||||
- **Inbound Rules**:
|
||||
- Port 22 (SSH): From admin IP only (for management)
|
||||
- Port 80 (HTTP): From 0.0.0.0/0 (redirects to HTTPS)
|
||||
- Port 443 (HTTPS): From 0.0.0.0/0 (Gitea access)
|
||||
- **Outbound Rules**:
|
||||
- All traffic: To 0.0.0.0/0 (for updates, backups to S3)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **SSH Access**: Restricted to specific admin IP address (your IP)
|
||||
2. **HTTP/HTTPS**: Open to internet (required for Gitea web access)
|
||||
3. **No Direct Gitea Access**: Port 3000 not exposed; only nginx on 80/443
|
||||
4. **Outbound**: Allowed for Docker image pulls, package updates, S3 backups
|
||||
|
||||
## Traffic Flow
|
||||
|
||||
1. User request → Internet → Internet Gateway
|
||||
2. Internet Gateway → Public Subnet → EC2 instance
|
||||
3. Security Group inspects and allows traffic on ports 80/443
|
||||
4. Nginx receives request, terminates SSL, proxies to Gitea
|
||||
Loading…
Reference in New Issue
Block a user