gitlab template updates

AGENTS.md

@@ -72,20 +72,28 @@ Example `template.yaml`:
 
 ```yaml
 ---
-kind: "compose"
+kind: compose
 metadata:
-  name: "My Nginx Template"
-  description: "A template for a simple Nginx service."
-  version: "0.1.0"
-  author: "Christian Lempa"
-  date: "2024-10-01"
+  name: My Nginx Template
+  description: >
+    A template for a simple Nginx service.
+
+
+    Project: https://...
+
+    Source: https://
+
+    Documentation: https://
+  version: 0.1.0
+  author: Christian Lempa
+  date: '2024-10-01'
 spec:
   general:
     vars:
       nginx_version:
-        type: "string"
-        description: "The Nginx version to use."
-        default: "latest"
+        type: string
+        description: The Nginx version to use.
+        default: latest
 ```
 
 #### Template Files
@@ -167,6 +175,7 @@ The `Variable.origin` attribute is updated to reflect this chain (e.g., `module
 
 - A section can be made conditional by setting the `toggle` property to the name of a boolean variable within that same section.
 - **Example**: `toggle: "advanced_enabled"`
+- **Validation**: The toggle variable MUST be of type `bool`. This is validated at load-time by `VariableCollection._validate_section_toggle()`.
 - During an interactive session, the CLI will first ask the user to enable or disable the section by prompting for the toggle variable (e.g., "Enable advanced settings?").
 - If the section is disabled (the toggle is `false`), all other variables within that section are skipped, and the section is visually dimmed in the summary table. This provides a clean way to manage optional or advanced configurations.
 
@@ -194,7 +203,7 @@ After creating the issue, update the TODO line in the `AGENTS.md` file with the
 
 ### Work in Progress
 
-* FIXME We need proper validation to ensure all variable names are unique across all sections
+* FIXME We need proper validation to ensure all variable names are unique across all sections (currently allowed but could cause conflicts)
 * FIXME Insufficient Error Messages for Template Loading
 * FIXME Excessive Generic Exception Catching
 * FIXME No Rollback on Config Write Failures: If writing config fails partway through, the config file can be left in a corrupted state. There's no atomic write operation.
@@ -208,3 +217,40 @@ After creating the issue, update the TODO line in the `AGENTS.md` file with the
 * TODO Template Validation Command: A command to validate the structure and variable definitions of a template without generating it.
 * TODO Interactive Variable Prompt Improvements: The interactive prompt could be improved with better navigation, help text, and validation feedback.
 * TODO Better Error Recovery in Jinja2 Rendering
+
+## Best Practices for Template Development
+
+### Template Structure
+- Always include a main `template.yaml` or `template.yml` file
+- Use descriptive template IDs (directory names) with lowercase and hyphens
+- Use dot notation for sub-templates (e.g., `parent.sub-name`)
+- Place Jinja2 templates in subdirectories when appropriate (e.g., `config/`)
+
+### Variable Definitions
+- Define variables in module specs for common, reusable settings
+- Define variables in template specs for template-specific settings
+- Only override specific fields in templates (don't redefine entire variables)
+- Use descriptive variable names with underscores (e.g., `external_url`, `smtp_port`)
+- Always specify `type` for new variables
+- Provide sensible `default` values when possible
+
+### Jinja2 Templates
+- Use `.j2` extension for all Jinja2 template files
+- Use conditional blocks (`{% if %}`) for optional features
+- Keep template logic simple and readable
+- Use comments to explain complex logic
+- Test with different variable combinations
+
+### Module Specs
+- Define common sections that apply to all templates of that kind
+- Use toggle variables for optional sections
+- Provide comprehensive descriptions for user guidance
+- Group related variables into logical sections
+- Validate toggle variables are boolean type
+
+### Testing Templates
+- Test generation with default values
+- Test with toggle sections enabled and disabled
+- Test with edge cases (empty values, special characters)
+- Verify yamllint compliance for YAML files
+- Check that generated files are syntactically valid

cli/core/variables.py

@@ -44,6 +44,9 @@ class Variable:
     if "name" not in data:
       raise ValueError("Variable data must contain 'name' key")
     
+    # Track which fields were explicitly provided in source data
+    self._explicit_fields: Set[str] = set(data.keys())
+    
     # Initialize fields
     self.name: str = data["name"]
     self.description: Optional[str] = data.get("description") or data.get("display", "")
@@ -359,7 +362,15 @@ class Variable:
     if update:
       data.update(update)
     
-    return Variable(data)
+    # Create new variable
+    cloned = Variable(data)
+    
+    # Preserve explicit fields from original, and add any update keys
+    cloned._explicit_fields = self._explicit_fields.copy()
+    if update:
+      cloned._explicit_fields.update(update.keys())
+    
+    return cloned
   
   # !SECTION
 
@@ -582,6 +593,39 @@ class VariableCollection:
       section.variables[var_name] = variable
       # NOTE: Populate the direct lookup map for efficient access.
       self._variable_map[var_name] = variable
+    
+    # Validate toggle variable after all variables are added
+    self._validate_section_toggle(section)
+    # FIXME: Add more section-level validation here as needed:
+    #   - Validate that variable names don't conflict across sections (currently allowed but could be confusing)
+    #   - Validate that required sections have at least one non-toggle variable
+    #   - Validate that enum variables have non-empty options lists
+    #   - Validate that variable names follow naming conventions (e.g., lowercase_with_underscores)
+    #   - Validate that default values are compatible with their type definitions
+
+  def _validate_section_toggle(self, section: VariableSection) -> None:
+    """Validate that toggle variable exists and is of type bool.
+    
+    Args:
+        section: The section to validate
+        
+    Raises:
+        ValueError: If toggle variable doesn't exist or is not boolean type
+    """
+    if not section.toggle:
+      return
+    
+    toggle_var = section.variables.get(section.toggle)
+    if not toggle_var:
+      raise ValueError(
+        f"Section '{section.key}' has toggle '{section.toggle}' but variable does not exist"
+      )
+    
+    if toggle_var.type != "bool":
+      raise ValueError(
+        f"Section '{section.key}' toggle variable '{section.toggle}' must be type 'bool', "
+        f"but is type '{toggle_var.type}'"
+      )
 
   # -------------------------
   # SECTION: Public API Methods
@@ -796,21 +840,21 @@ class VariableCollection:
         # Variable exists in both - merge with other taking precedence
         self_var = merged_section.variables[var_name]
         
-        # Build update dict with other's values taking precedence
+        # Build update dict with ONLY explicitly provided fields from other
         update = {}
-        if other_var.type:
+        if 'type' in other_var._explicit_fields and other_var.type:
           update['type'] = other_var.type
-        if other_var.value is not None:
+        if ('value' in other_var._explicit_fields or 'default' in other_var._explicit_fields) and other_var.value is not None:
           update['value'] = other_var.value
-        if other_var.description:
+        if 'description' in other_var._explicit_fields and other_var.description:
           update['description'] = other_var.description
-        if other_var.prompt:
+        if 'prompt' in other_var._explicit_fields and other_var.prompt:
           update['prompt'] = other_var.prompt
-        if other_var.options:
+        if 'options' in other_var._explicit_fields and other_var.options:
           update['options'] = other_var.options
-        if other_var.sensitive:
+        if 'sensitive' in other_var._explicit_fields and other_var.sensitive:
           update['sensitive'] = other_var.sensitive
-        if other_var.extra:
+        if 'extra' in other_var._explicit_fields and other_var.extra:
           update['extra'] = other_var.extra
         
         # Update origin tracking (only keep the current source, not the chain)

cli/modules/compose.py

@@ -69,7 +69,7 @@ spec = OrderedDict(
           "ports_enabled": {
             "description": "Expose ports via 'ports' mapping",
             "type": "bool",
-            "default": False,
+            "default": True,
           }
         },
       },
@@ -84,6 +84,11 @@ spec = OrderedDict(
             "type": "bool",
             "default": False,
           },
+          "traefik_network": {
+            "description": "Traefik network name",
+            "type": "str",
+            "default": "traefik",
+          },
           "traefik_host": {
             "description": "Domain name for your service",
             "type": "hostname",
@@ -205,6 +210,35 @@ spec = OrderedDict(
           },
         },
       },
+      "authentik": {
+        "title": "Authentik SSO",
+        "prompt": "Configure Authentik SSO integration?",
+        "toggle": "authentik_enabled",
+        "description": "Single Sign-On using Authentik identity provider.",
+        "vars": {
+          "authentik_enabled": {
+            "description": "Enable Authentik SSO integration",
+            "type": "bool",
+            "default": False,
+          },
+          "authentik_url": {
+            "description": "Authentik base URL (e.g., https://auth.example.com)",
+            "type": "str",
+          },
+          "authentik_slug": {
+            "description": "Authentik application slug",
+            "type": "str",
+          },
+          "authentik_client_id": {
+            "description": "OAuth client ID from Authentik provider",
+            "type": "str",
+          },
+          "authentik_client_secret": {
+            "description": "OAuth client secret from Authentik provider",
+            "type": "str",
+          },
+        },
+      },
     }
   )
 

library/compose/alloy/template.yaml

@@ -2,14 +2,17 @@
 kind: compose
 metadata:
   name: Grafana Alloy
-  description: |
-    Grafana Alloy is an open telemetry collector distribution that gathers and
-    processes logs, metrics, traces and profiles. It combines features from the
-    OpenTelemetry Collector and Prometheus and provides programmable pipelines
-    and high-performance, vendor-neutral observability.
+  description: >
+    Grafana Alloy is an open telemetry collector distribution that gathers
+    and processes logs, metrics, traces and profiles. It combines features
+    from the OpenTelemetry Collector and Prometheus and provides programmable
+    pipelines and high-performance, vendor-neutral observability.
+
 
     Project: https://grafana.com/docs/alloy/
+
     Source: https://github.com/grafana/alloy
+
     Documentation: https://grafana.com/docs/alloy/latest/
   version: 0.0.1
   author: Christian Lempa

library/compose/gitlab/compose.yaml.j2

@@ -1,51 +1,58 @@
 services:
-  gitlab:
+  {{ service_name | default('gitlab') }}:
     image: docker.io/gitlab/gitlab-ce:18.3.1-ce.0
-    container_name: gitlab
+    container_name: {{ container_name | default('gitlab') }}
     shm_size: '256m'
-    environment: {}
-    # --> (Optional) When using traefik...
-    # networks:
-    #   - frontend
-    # <--
+{% if traefik_enabled %}
+    networks:
+      - {{ traefik_network }}
+{% endif %}
     volumes:
-      - ./config:/etc/gitlab
-      - ./logs:/var/log/gitlab
+      - ./config/gitlab.rb:/etc/gitlab/gitlab.rb:ro
+      - gitlab-config:/etc/gitlab
+      - gitlab-logs:/var/log/gitlab
       - gitlab-data:/var/opt/gitlab
+{% if ports_enabled %}
     ports:
-      # --> (Optional) Remove when using traefik...
-      - "80:80"
-      - "443:443"
-      # <--
-      - '2424:22'
-    # --> (Optional) When using traefik...
-    # labels:
-    #   - traefik.enable=true
-    #   - traefik.http.services.gitlab.loadbalancer.server.port=80
-    #   - traefik.http.services.gitlab.loadbalancer.server.scheme=http
-    #   - traefik.http.routers.gitlab.service=gitlab
-    #   - traefik.http.routers.gitlab.rule=Host(`your-gitlab-fqdn`)
-    #   - traefik.http.routers.gitlab.entrypoints=websecure
-    #   - traefik.http.routers.gitlab.tls=true
-    #   - traefik.http.routers.gitlab.tls.certresolver=cloudflare
-    # <--
-    # --> (Optional) Enable Container Registry settings here...
-    #   - traefik.http.services.registry.loadbalancer.server.port=5678
-    #   - traefik.http.services.registry.loadbalancer.server.scheme=http
-    #   - traefik.http.routers.registry.service=registry
-    #   - traefik.http.routers.registry.rule=Host(`your-registry-fqdn`)
-    #   - traefik.http.routers.registry.entrypoints=websecure
-    #   - traefik.http.routers.registry.tls=true
-    #   - traefik.http.routers.registry.tls.certresolver=cloudflare
-    # <--
-    restart: unless-stopped
+      - "{{ ports_http }}:80"
+      - "{{ ports_https }}:443"
+      - "{{ ssh_port }}:22"
+{% else %}
+    ports:
+      - "{{ ssh_port }}:22"
+{% endif %}
+{% if traefik_enabled %}
+    labels:
+      - traefik.enable=true
+      - traefik.http.services.{{ container_name }}.loadbalancer.server.port=80
+      - traefik.http.services.{{ container_name }}.loadbalancer.server.scheme=http
+      - traefik.http.routers.{{ container_name }}.service={{ container_name }}
+      - traefik.http.routers.{{ container_name }}.rule=Host(`{{ traefik_host }}`)
+      - traefik.http.routers.{{ container_name }}.entrypoints={{ traefik_tls_entrypoint }}
+      - traefik.http.routers.{{ container_name }}.tls=true
+      - traefik.http.routers.{{ container_name }}.tls.certresolver={{ traefik_tls_certresolver }}
+{% if registry_enabled %}
+      - traefik.http.services.{{ container_name }}-registry.loadbalancer.server.port={{ registry_port }}
+      - traefik.http.services.{{ container_name }}-registry.loadbalancer.server.scheme=http
+      - traefik.http.routers.{{ container_name }}-registry.service={{ container_name }}-registry
+      - traefik.http.routers.{{ container_name }}-registry.rule=Host(`{{ registry_hostname }}`)
+      - traefik.http.routers.{{ container_name }}-registry.entrypoints={{ traefik_tls_entrypoint }}
+      - traefik.http.routers.{{ container_name }}-registry.tls=true
+      - traefik.http.routers.{{ container_name }}-registry.tls.certresolver={{ traefik_tls_certresolver }}
+{% endif %}
+{% endif %}
+    restart: {{ restart_policy }}
 
 volumes:
+  gitlab-config:
+    driver: local
+  gitlab-logs:
+    driver: local
   gitlab-data:
     driver: local
 
-# --> (Optional) When using traefik...
-# networks:
-#   frontend:
-#     external: true
-# <--
+{% if traefik_enabled %}
+networks:
+  {{ traefik_network }}:
+    external: true
+{% endif %}

library/compose/gitlab/config/gitlab.rb

@@ -1,58 +0,0 @@
-# -- Change GitLab settings here...
-external_url 'https://your-gitlab-fqdn'  # <-- Replace with your GitLab FQDN
-
-# -- (Optional) Change GitLab Shell settings here...
-gitlab_rails['gitlab_shell_ssh_port'] = 2424
-
-# -- Change internal web service settings here...
-letsencrypt['enable'] = false
-nginx['listen_port']  = 80
-nginx['listen_https'] = false
-
-# --> (Optional) Enable Container Registry settings here...
-# registry_external_url 'https://your-registry-fqdn'  # <-- Replace with your registry FQDN
-# gitlab_rails['registry_enabled']  = true
-# registry_nginx['listen_https']    = false
-# registry_nginx['listen_port']     = 5678  # <-- Replace with your registry port
-# <--
-
-# --> (Optional) Add Authentik settings here...
-# gitlab_rails['omniauth_auto_link_user'] = ['openid_connect']
-# gitlab_rails['omniauth_providers'] = [
-#   {
-#     name: "openid_connect",  #  !-- Do not change this parameter
-#     label: "Authentik",  # <-- (Optional) Change name for login button, defaults to "Openid Connect"
-#     icon: "https://avatars.githubusercontent.com/u/82976448?s=200&v=4",
-#     args: {
-#       name: "openid_connect",
-#       scope: ["openid","profile","email"],
-#       response_type: "code",
-#       issuer: "https://your-authentik-fqdn/application/o/your-gitlab-slug/",  # <-- Replace with your Authentik FQDN and GitLab slug
-#       discovery: true,
-#       client_auth_method: "query",
-#       uid_field: "email",
-#       send_scope_to_token_endpoint: "false",
-#       pkce: true,
-#       client_options: {
-#         identifier: "your-authentik-provider-client-id",  # <-- Replace with your Authentik provider client ID
-#         secret: "your-authentik-provider-client-secret",  # <-- Replace with your Authentik provider client secret
-#         redirect_uri: "https://your-authentik-fqdn/users/auth/openid_connect/callback"  # <-- Replace with your Authentik FQDN
-#       }
-#     }
-#   }
-# ]
-# <--
-
-# --> (Optional) Change SMTP settings here...
-# gitlab_rails['smtp_enable']           = true
-# gitlab_rails['smtp_address']          = "your-smtp-server-addr"  # <-- Replace with your SMTP server address
-# gitlab_rails['smtp_port']             = 465
-# gitlab_rails['smtp_user_name']        = "your-smtp-username"  # <-- Replace with your SMTP username
-# gitlab_rails['smtp_password']         = "your-smtp-password"  # <-- Replace with your SMTP password
-# gitlab_rails['smtp_domain']           = "your-smtp-domain"  # <-- Replace with your SMTP domain
-# gitlab_rails['smtp_authentication']   = "login"
-# gitlab_rails['smtp_ssl']              = true
-# gitlab_rails['smtp_force_ssl']        = true
-# gitlab_rails['gitlab_email_from']     = 'your-email-from-addr'  # <-- Replace with your email from address
-# gitlab_rails['gitlab_email_reply_to'] = 'your-email-replyto-addr'  # <-- Replace with your email reply-to address
-# <--

library/compose/gitlab/config/gitlab.rb.j2

@@ -0,0 +1,72 @@
+# GitLab Configuration
+external_url '{{ external_url }}'
+
+# GitLab Shell SSH settings
+gitlab_rails['gitlab_shell_ssh_port'] = {{ ssh_port }}
+
+# Internal web service settings
+{% if traefik_enabled %}
+# Traefik handles TLS/SSL certificates
+letsencrypt['enable'] = false
+nginx['listen_port']  = 80
+nginx['listen_https'] = false
+{% else %}
+# Let's Encrypt certificate management (when not using Traefik)
+letsencrypt['enable'] = true
+letsencrypt['contact_emails'] = ['{{ email_from|default("admin@example.com") }}']
+nginx['redirect_http_to_https'] = true
+{% endif %}
+
+{% if registry_enabled %}
+# Container Registry settings
+registry_external_url '{{ registry_external_url }}'
+gitlab_rails['registry_enabled']  = true
+registry_nginx['listen_https']    = false
+registry_nginx['listen_port']     = {{ registry_port }}
+{% endif %}
+
+{% if authentik_enabled %}
+# Authentik SSO settings
+gitlab_rails['omniauth_auto_link_user'] = ['openid_connect']
+gitlab_rails['omniauth_providers'] = [
+  {
+    name: "openid_connect",
+    label: "Authentik",
+    icon: "https://avatars.githubusercontent.com/u/82976448?s=200&v=4",
+    args: {
+      name: "openid_connect",
+      scope: ["openid","profile","email"],
+      response_type: "code",
+      issuer: "{{ authentik_url }}/application/o/{{ authentik_slug }}/",
+      discovery: true,
+      client_auth_method: "query",
+      uid_field: "email",
+      send_scope_to_token_endpoint: "false",
+      pkce: true,
+      client_options: {
+        identifier: "{{ authentik_client_id }}",
+        secret: "{{ authentik_client_secret }}",
+        redirect_uri: "{{ external_url }}/users/auth/openid_connect/callback"
+      }
+    }
+  }
+]
+{% endif %}
+
+{% if email_enabled %}
+# SMTP settings
+gitlab_rails['smtp_enable']           = true
+gitlab_rails['smtp_address']          = "{{ email_host }}"
+gitlab_rails['smtp_port']             = {{ email_port }}
+gitlab_rails['smtp_user_name']        = "{{ email_username }}"
+gitlab_rails['smtp_password']         = "{{ email_password }}"
+gitlab_rails['smtp_authentication']   = "login"
+{% if email_use_ssl %}
+gitlab_rails['smtp_ssl']              = true
+gitlab_rails['smtp_force_ssl']        = true
+{% elif email_use_tls %}
+gitlab_rails['smtp_tls']              = true
+{% endif %}
+gitlab_rails['gitlab_email_from']     = '{{ email_from }}'
+gitlab_rails['gitlab_email_reply_to'] = '{{ email_from }}'
+{% endif %}

library/compose/gitlab/template.yaml

@@ -1,21 +1,65 @@
 ---
 kind: compose
 metadata:
-  name: Gitlab
-  description: Docker compose setup for gitlab
+  name: GitLab
+  description: >
+    GitLab is a web-based DevOps lifecycle tool that provides a Git repository
+    manager providing wiki, issue-tracking, and CI/CD pipeline features, using
+    an open-source license, developed by GitLab Inc.
+
+    Project: https://about.gitlab.com/
+
+    Source: https://gitlab.com/gitlab-org/gitlab
+
+    Documentation: https://docs.gitlab.com/
   version: 0.1.0
   author: Christian Lempa
   date: '2025-09-28'
   tags:
-  - gitlab
-  - docker
-  - compose
+    - git
+    - devops
+    - ci-cd
 spec:
   general:
     vars:
-      gitlab_version:
+      external_url:
         type: string
-        description: Gitlab version
-        default: latest
-
----
+        description: External URL for GitLab (e.g., https://gitlab.example.com)
+        default: 'https://gitlab.example.com'
+      ssh_port:
+        type: int
+        description: SSH port for Git operations
+        default: 2424
+  ports:
+    vars:
+      ports_enabled:
+        description: Expose HTTP/HTTPS ports (disabled if using Traefik)
+      ports_http:
+        type: int
+        description: HTTP port (disabled if using Traefik)
+        default: 80
+      ports_https:
+        type: int
+        description: HTTPS port (disabled if using Traefik)
+        default: 443
+  registry:
+    description: GitLab Container Registry configuration
+    required: false
+    toggle: registry_enabled
+    vars:
+      registry_enabled:
+        type: bool
+        description: Enable GitLab Container Registry
+        default: false
+      registry_external_url:
+        type: string
+        description: External URL for Container Registry
+        default: 'https://registry.example.com'
+      registry_hostname:
+        type: string
+        description: Hostname for Container Registry (when using Traefik)
+        default: registry.example.com
+      registry_port:
+        type: int
+        description: Internal port for Container Registry
+        default: 5678

test/compose.yaml

@@ -0,0 +1,24 @@
+services:
+  gitlab:
+    image: docker.io/gitlab/gitlab-ce:18.3.1-ce.0
+    container_name: gitlab
+    shm_size: '256m'
+    volumes:
+      - ./config/gitlab.rb:/etc/gitlab/gitlab.rb:ro
+      - gitlab-config:/etc/gitlab
+      - gitlab-logs:/var/log/gitlab
+      - gitlab-data:/var/opt/gitlab
+    ports:
+      - "80:80"
+      - "443:443"
+      - "2424:22"
+    restart: unless-stopped
+
+volumes:
+  gitlab-config:
+    driver: local
+  gitlab-logs:
+    driver: local
+  gitlab-data:
+    driver: local
+

test/config/gitlab.rb

@@ -0,0 +1,14 @@
+# GitLab Configuration
+external_url 'https://gitlab.example.com'
+
+# GitLab Shell SSH settings
+gitlab_rails['gitlab_shell_ssh_port'] = 2424
+
+# Internal web service settings
+# Let's Encrypt certificate management (when not using Traefik)
+letsencrypt['enable'] = true
+letsencrypt['contact_emails'] = ['admin@example.com']
+nginx['redirect_http_to_https'] = true
+
+
+