Changes between Version 1 and Version 2 of OMFServices Deployment


Ignore:
Timestamp:
Mar 20, 2026, 10:51:59 PM (7 days ago)
Author:
editor
Comment:

Updating ORBIT/COSMOS service deployment topology

Legend:

Unmodified
Added
Removed
Modified

  • OMFServices Deployment

    v1 v2  
    1 = OMF Services Deployment Topology =
     1= ORBIT/COSMOS Service Deployment Topology =
    22
    3 '''Last updated: 2026-03-13'''
     3== Overview ==
     4Two-site testbed architecture:
     5 * '''North Brunswick, NJ''' (Rutgers University): IPs 10.0.0.0 - 10.63.255.255
     6 * '''New York City, NY''' (Columbia University): IPs 10.64.0.0 - 10.127.255.255
     7 * Connected via IP tunnel for management traffic
    48
    5 [[PageOutline(2-3)]]
     919 Ruby/Sinatra microservices + 1 Python/Flask service + 1 CLI tool (omf-expctl) manage all testbed operations.
    610
    7 == 1. Overview ==
     11== North Brunswick Hosts ==
    812
    9 The ORBIT/COSMOS testbed infrastructure spans two sites connected via IP tunnel:
     13=== am1 (10.50.0.41) — Aggregate Manager Proxy ===
     14|| '''Service''' || '''Port''' || '''Version''' || '''Description''' ||
     15|| omf-agg-mgr-proxy || 5054 || v1-1 || Service discovery, API proxy ||
     16|| omf-rf-control || 5001 || v0-2 || RF attenuator control ||
     17|| omf-rf-switch || 5002 || v0-2 || RF switch matrix control ||
     18|| omf-xy-table || 5003 || - || XY table proxy (→ RPi controllers) ||
     19|| omf-array-mgmt || 5004 || - || Antenna array management (Python/Flask) ||
    1020
    11  * '''North Brunswick, NJ (Rutgers)''' -- Primary ORBIT site
    12  * '''New York City, NY (Columbia)''' -- Primary COSMOS site
     21=== am4 — Development Server ===
     22 * All service source repositories at /home/seskar/omf-*
     23 * Build host for all .deb packages
     24 * Not running production services
    1325
    14 The platform runs '''19 Ruby/Sinatra microservices''', '''1 Python/Flask service''' (omf-array-mgmt), and '''1 CLI tool''' (omf-expctl) across '''23+ hosts''' with '''35+ service instances'''.
     26=== am5 (10.50.0.45) — Core Services ===
     27|| '''Service''' || '''Port''' || '''Version''' || '''Description''' ||
     28|| omf-cmc || 5013 || v1-9 || Chassis management (IPMI, HTTP CM, PDU) ||
     29|| omf-scheduler || 5016 || v1-1 || Reservation scheduler with LDAP host management ||
     30|| omf-rfmatrix || 5020 || v1-2 || RF attenuator matrix control ||
     31|| omf-status || 5021 || v1-1 || Testbed node power status aggregation ||
    1532
    16 All Ruby services share the {{{omf-common}}} git submodule (Sinatra base class with DSL for route definition, XML/JSON response formatting, Prometheus metrics, and configuration loading). Services are packaged as Debian packages via FPM and managed by systemd.
     33=== repository2 (10.50.0.22) — Imaging & User Services ===
     34|| '''Service''' || '''Port''' || '''Version''' || '''Description''' ||
     35|| omf-account-mgmt || 5017 || v1-3 || User/group registration, LDAP lifecycle ||
     36|| omf-frisbee || 5011 || v1-4 || Frisbee multicast daemon management ||
     37|| omf-pxe || 5010 || v1-6 || PXE boot configuration ||
     38|| omf-saveimage || 5012 || v1-4 || Disk image capture via netcat ||
     39|| omf-user-stats || 5015 || v1-4 || Usage statistics (disk, scheduler, imaging) ||
     40|| iso-server || 8080 || - || ISO image HTTP server ||
    1741
    18 The canonical entry point for all service API calls is the '''AM proxy''' at {{{am1:5054}}} ({{{omf-agg-mgr-proxy}}}), which provides service discovery and request routing. The '''cosmos-portal''' (React SPA on web1) provides the web UI, with Apache reverse-proxying API calls to backend services.
     42=== mgmt1 (10.250.0.8) / mgmt2 (10.250.0.9) — Network Infrastructure ===
     43 * '''DHCP''': ISC DHCP Server 4, 2,145 static hosts, failover pair
     44 * '''DNS''': BIND 9, 2,563 A records, 25 forward zones, 53 reverse zones
     45 * '''Kea DHCP''': Installed but stopped (migration pending)
    1946
    20 == 2. Network Architecture ==
     47=== Other NJ Infrastructure ===
     48|| '''Host''' || '''Service''' || '''Purpose''' ||
     49|| web1 || cosmos-portal || React SPA + Apache reverse proxy ||
     50|| db1 (10.0.0.51) || LibreNMS || Network monitoring (190 devices) ||
     51|| mysql1 || MySQL || Shared database server ||
     52|| amqp.orbit-lab.org || RabbitMQ || MQTT broker for EC↔RC communication ||
     53|| gitlab.orbit-lab.org || GitLab || Source code repository ||
    2154
    22 === IP Address Ranges ===
     55=== ORBIT Console Servers (9) ===
     56Each runs omf-cmonitor (port 5000) + omf-expctl:
     57|| '''Console''' || '''Domain''' ||
     58|| grid.orbit-lab.org || 20x20 indoor grid ||
     59|| sb1.orbit-lab.org || Sandbox 1 ||
     60|| sb2.orbit-lab.org || Sandbox 2 ||
     61|| sb3.orbit-lab.org || Sandbox 3 ||
     62|| sb4.orbit-lab.org || Sandbox 4 ||
     63|| sb7.orbit-lab.org || Sandbox 7 ||
     64|| sb9.orbit-lab.org || Sandbox 9 ||
     65|| outdoor.orbit-lab.org || Outdoor network ||
     66|| instrument.orbit-lab.org || Instrumentation ||
    2367
    24 ||= Site =||= Range =||= Notes =||
    25 || North Brunswick (Rutgers) || {{{10.0.0.0 -- 10.63.255.255}}} || Primary ORBIT infrastructure ||
    26 || New York City (Columbia) || {{{10.64.0.0 -- 10.127.255.255}}} || Primary COSMOS infrastructure ||
     68== New York City Hosts ==
    2769
    28 === VLAN Structure ===
     70=== am-crf.cosmos-lab.org — NYC Aggregate Manager ===
     71|| '''Service''' || '''Port''' || '''Version''' || '''Description''' ||
     72|| omf-agg-mgr-proxy || 5054 || v1-1 || Local AM proxy (mirrors am1) ||
     73|| omf-frisbee || 5011 || v1-4 || Local frisbee (127.0.0.1) ||
     74|| omf-pxe || 5010 || v1-6 || Local PXE (127.0.0.1) ||
     75|| omf-saveimage || 5012 || v1-4 || Local saveimage (127.0.0.1) ||
    2976
    30  * '''Management VLAN''' -- Infrastructure servers, out-of-band management
    31  * '''Control VLAN''' (per domain) -- Service-to-node communication, PXE boot
    32  * '''Data VLAN''' (per domain) -- Experiment data traffic between testbed nodes
    33  * '''IP tunnel''' connecting North Brunswick and NYC sites
     77=== COSMOS Console Servers (9) ===
     78Each runs omf-cmonitor (port 5000) + omf-expctl:
     79|| '''Console''' || '''Domain''' ||
     80|| osc.cosmos-lab.org || Open Square Campus ||
     81|| indigo.cosmos-lab.org || Indigo ||
     82|| accord.cosmos-lab.org || ACCORD ||
     83|| sb1.cosmos-lab.org || COSMOS Sandbox 1 ||
     84|| sb2.cosmos-lab.org || COSMOS Sandbox 2 ||
     85|| weeks.cosmos-lab.org || Weeks ||
     86|| rrail.cosmos-lab.org || RRAIL ||
     87|| bed.cosmos-lab.org || BED ||
     88|| nebula.cosmos-lab.org || Nebula ||
    3489
    35 === DNS Domains ===
     90=== Specialized NYC Hardware ===
     91|| '''Host''' || '''Service''' || '''Port''' || '''Description''' ||
     92|| pi1-auden.sb1.cosmos-lab.org || omf-cosmos-cm, omf-auden || 5018, 5019 || Radio not responding (HW issue) ||
     93|| pi2-auden.sb1.cosmos-lab.org || omf-cosmos-cm, omf-auden || 5018, 5019 || Working, auto-recovery ||
     94|| xytable1.sb1.cosmos-lab.org || omf-xytable-ctrl || 80 || XY table controller (RPi) ||
     95|| xytable2.sb1.cosmos-lab.org || omf-xytable-ctrl || 80 || XY table controller (RPi) ||
    3696
    37  * {{{orbit-lab.org}}} -- ORBIT nodes and infrastructure
    38  * {{{cosmos-lab.org}}} -- COSMOS nodes and infrastructure
     97== Service Dependency Map ==
    3998
    40 DNS is served by BIND9 on mgmt1/mgmt2 with 2,563 forward A records across 25 zones (11 ORBIT + 14 COSMOS) and 53 reverse zones.
     99{{{
     100                    ┌─────────────────────┐
     101                    │   cosmos-portal     │
     102                    │     (web1)          │
     103                    └──────────┬──────────┘
     104                               │ Apache reverse proxy
     105                    ┌──────────▼──────────┐
     106                    │  omf-agg-mgr-proxy  │
     107                    │  am1:5054 / am-crf  │
     108                    └──────────┬──────────┘
     109           ┌───────────────────┼───────────────────┐
     110           │                   │                   │
     111    ┌──────▼──────┐    ┌──────▼──────┐    ┌──────▼──────┐
     112    │ omf-cmc     │    │ omf-frisbee │    │ omf-pxe     │
     113    │ am5:5013    │    │ repos:5011  │    │ repos:5010  │
     114    └─────────────┘    └─────────────┘    └─────────────┘
    41115
    42 == 3. North Brunswick Hosts ==
     116    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
     117    │ omf-sched   │    │ omf-acct    │    │ omf-status  │
     118    │ am5:5016    │    │ repos:5017  │    │ am5:5021    │
     119    └──────┬──────┘    └──────┬──────┘    └─────────────┘
     120           │                  │
     121     ┌─────▼─────┐     ┌─────▼─────┐
     122     │  MySQL    │     │   LDAP    │
     123     │  mysql1   │     │   ldap1   │
     124     └───────────┘     └───────────┘
     125}}}
    43126
    44 === am1 (10.50.0.41) -- Aggregate Manager / RF Services ===
     127== Port Registry ==
    45128
    46 ||= Service =||= Port =||= Version =||= Description =||
    47 || omf-agg-mgr-proxy || 5054 || -- || Service discovery proxy (routes API calls to backend services) ||
    48 || omf-rf-control || 5001 || v0-2 || RF signal generator control ||
    49 || omf-rf-switch || 5002 || v0-2 || RF switch matrix control ||
    50 || omf-xy-table || 5003 || -- || XY table positioning service ||
    51 || omf-array-mgmt || 5004 || -- || Antenna array management ('''Python/Flask''') ||
    52 
    53 '''Platform:''' Ubuntu 18.04, RVM Ruby 3.2.3, Python 3.6.9[[BR]]
    54 '''Note:''' Requires {{{libruby-3.2}}} symlink + ldconfig for native extensions
    55 
    56 === am4 -- Development Server ===
    57 
    58 Development machine only. All source repos at {{{/home/seskar/omf-*}}}. Not a production service host.
    59 
    60 === am5 (10.50.0.45) -- Core Services ===
    61 
    62 ||= Service =||= Port =||= Version =||= Description =||
    63 || omf-cmc || 5013 || v1-8 || Chassis management controller (power control via IPMI, HTTP CM, SNMP PDU) ||
    64 || omf-scheduler || 5016 || -- || Reservation scheduler and auto-approver (ActiveRecord + MySQL) ||
    65 || omf-rfmatrix || 5020 || v1-1 || RF matrix switch control ||
    66 || omf-status || 5021 || -- || Testbed status aggregation ||
    67 
    68 === repository2 (10.50.0.22) -- Image & Account Services ===
    69 
    70 ||= Service =||= Port =||= Version =||= Description =||
    71 || omf-account-mgmt || 5017 || v1-2 || User/group registration, approval, LDAP lifecycle (ActiveRecord + MySQL) ||
    72 || omf-frisbee || 5011 || v1-3 || Frisbee daemon management for disk image multicasting ||
    73 || omf-pxe || 5010 || v1-5 || PXE boot configuration (aggregate manager) ||
    74 || omf-saveimage || 5012 || v1-3 || Disk image save via netcat receiver ||
    75 || omf-user-stats || 5015 || -- || User disk usage and scheduler usage statistics ||
    76 
    77 === Infrastructure Servers ===
    78 
    79 ||= Host =||= IP =||= Role =||
    80 || mgmt1 || 10.250.0.8 || Primary DHCP (ISC DHCP4, 2,145 static hosts) + Primary DNS (BIND9, 2,563 A records) ||
    81 || mgmt2 || 10.250.0.9 || DHCP failover peer + DNS slave ||
    82 || db1 || 10.0.0.51 || LibreNMS monitoring (190 devices, SNMP polling every 5 min) ||
    83 || mysql1 || -- || Shared MySQL server for scheduler, account-mgmt, user-stats ||
    84 || amqp.orbit-lab.org || -- || RabbitMQ MQTT broker (MQTT 1883, WebSocket 15675) ||
    85 || web1 || -- || cosmos-portal (React SPA), Apache reverse proxy ||
    86 || gitlab.orbit-lab.org || 10.50.0.20 || GitLab (24 repos under {{{orbit/}}}) ||
    87 || ldap1.orbit-lab.org || -- || Primary OpenLDAP server (port 389) ||
    88 || ldap2.orbit-lab.org || -- || Secondary OpenLDAP server (port 389) ||
    89 
    90 === ORBIT Console Servers (9 hosts) ===
    91 
    92 All consoles run '''omf-cmonitor''' on port 5000. Some also run '''omf-expctl'''.[[BR]]
    93 '''Platform:''' Ubuntu 16.04, RVM Ruby 3.2.3, omf-cmonitor v1-1
    94 
    95 ||= Console =||= omf-cmonitor =||= omf-expctl =||= Notes =||
    96 || grid.orbit-lab.org || Yes || Yes (v1-19) || Main 20x20 grid ||
    97 || sb1.orbit-lab.org || Yes || Yes (v1-19) || Sandbox 1 ||
    98 || sb2.orbit-lab.org || Yes || Yes (v1-19) || Sandbox 2 ||
    99 || sb3.orbit-lab.org || Yes || Yes (v1-19) || Sandbox 3 ||
    100 || sb4.orbit-lab.org || Yes || -- || Sandbox 4 ||
    101 || sb7.orbit-lab.org || Yes || -- || Sandbox 7 ||
    102 || sb9.orbit-lab.org || Yes || -- || Sandbox 9 ||
    103 || outdoor.orbit-lab.org || Yes || -- || Outdoor testbed ||
    104 || instrument.orbit-lab.org || Yes || -- || Instrument cluster ||
    105 
    106 === Unreachable Consoles ===
    107 
    108 The following consoles are currently unreachable: {{{vgrid1-4.orbit-lab.org}}}, {{{instrument.cosmos-lab.org}}}
    109 
    110 == 4. New York City Hosts ==
    111 
    112 === COSMOS Console Servers (9 hosts) ===
    113 
    114 All consoles run '''omf-cmonitor''' on port 5000.[[BR]]
    115 '''Platform:''' Ubuntu 16.04, RVM Ruby 3.2.3, omf-cmonitor v1-1
    116 
    117 ||= Console =||= omf-cmonitor =||= omf-expctl =||= Notes =||
    118 || osc.cosmos-lab.org || Yes || -- || Open-access sandbox ||
    119 || indigo.cosmos-lab.org || Yes || -- || ||
    120 || accord.cosmos-lab.org || Yes || -- || ||
    121 || sb1.cosmos-lab.org || Yes || Yes (v1-19) || Sandbox 1 ||
    122 || sb2.cosmos-lab.org || Yes || -- || Sandbox 2 ||
    123 || weeks.cosmos-lab.org || Yes || -- || ||
    124 || rrail.cosmos-lab.org || Yes || -- || ||
    125 || bed.cosmos-lab.org || Yes || -- || ||
    126 || nebula.cosmos-lab.org || Yes || -- || ||
    127 
    128 === COSMOS Raspberry Pis ===
    129 
    130 ||= Host =||= IP =||= Services =||
    131 || pi1-auden.sb1.cosmos-lab.org || 10.37.25.15 || omf-cosmos-cm (5018, v1-1), omf-auden (5019, v1-1) ||
    132 || pi2-auden.sb1.cosmos-lab.org || 10.37.25.16 || omf-cosmos-cm (5018, v1-1), omf-auden (5019, v1-1) ||
    133 
    134 === XY Table Controllers ===
    135 
    136 ||= Host =||= IP =||= Service =||= Notes =||
    137 || xytable1 || 10.1.37.221 || omf-xytable-ctrl (port 80) || Raspberry Pi, sb1.cosmos-lab.org ||
    138 || xytable2 || 10.1.37.222 || omf-xytable-ctrl (port 80) || Raspberry Pi, sb1.cosmos-lab.org ||
    139 
    140 MQTT telemetry published to {{{xy/<fqdn>/position}}} every 200ms via {{{amqp.orbit-lab.org:1883}}}.
    141 
    142 == 5. Shared Infrastructure ==
    143 
    144 ||= Component =||= Host =||= Details =||
    145 || cosmos-portal || web1 || React 18 + Vite SPA, Tailwind CSS, static files served by Apache ||
    146 || GitLab || gitlab.orbit-lab.org (10.50.0.20) || 24 repos under {{{orbit/}}} namespace ||
    147 || NetBox || 10.50.0.93 || v2.9.10 (needs upgrade to 4.x), 290 devices, data from 2020-2021 ||
    148 || Proxmox (ORBIT) || mgmt-vmhost1..5 || 5x Dell R740 (48 cores, 187GB each), 98 VMs (57 running), Ceph RBD + NFS ||
    149 || Proxmox (COSMOS) || mgmt-vmhost1..5-co1 || 3x R430 + 2x R740, 13 VMs (8 running) ||
    150 
    151 == 6. Service Dependency Map ==
    152 
    153 This section documents which services call which other services.
    154 
    155 === omf-expctl (Experiment Controller CLI) ===
    156 
    157 All calls routed via the AM proxy at {{{am1:5054}}}.
    158 
    159  * {{{omf-expctl}}} -> {{{omf-cmc}}} -- Power control (on/off/reset nodes)
    160  * {{{omf-expctl}}} -> {{{omf-pxe}}} -- PXE boot setup (set boot image)
    161  * {{{omf-expctl}}} -> {{{omf-frisbee}}} -- Disk imaging (load images onto nodes)
    162  * {{{omf-expctl}}} -> {{{omf-saveimage}}} -- Save disk images from nodes
    163  * {{{omf-expctl}}} -> {{{omf-scheduler}}} -- Permission/reservation check
    164 
    165 === Inter-Service Dependencies ===
    166 
    167 ||= Caller =||= Callee =||= Purpose =||= Via =||
    168 || omf-cmc || omf-cmonitor || Wake-on-LAN packet generation || Direct HTTP (CM_wolurl) ||
    169 || omf-auden || omf-rf-control || RF signal generator setup || Direct (should use AM proxy) ||
    170 || omf-status || omf-cmc || Node power state || Direct ||
    171 || omf-status || omf-scheduler || Reservation info || Direct ||
    172 || omf-status || omf-frisbee || Imaging status || Direct ||
    173 
    174 === External Dependencies ===
    175 
    176 ||= Service =||= External System =||= Purpose =||
    177 || omf-scheduler || LDAP (ldap1/ldap2) || Host attribute management (LdapHostManager) ||
    178 || omf-scheduler || MySQL || Reservation persistence ||
    179 || omf-scheduler || SMTP (mail.orbit-lab.org:25) || Reservation notifications ||
    180 || omf-account-mgmt || LDAP (ldap1/ldap2) || User/group lifecycle management ||
    181 || omf-account-mgmt || MySQL || Account persistence ||
    182 || omf-account-mgmt || SMTP (mail.orbit-lab.org:25) || Account notifications ||
    183 || omf-user-stats || MySQL (multiple databases) || Usage data aggregation ||
    184 || omf-user-stats || LDAP || User lookups ||
    185 
    186 === cosmos-portal (Web UI) ===
    187 
    188 All API calls proxied via Apache on web1:
    189 
    190 ||= Portal Route =||= Backend =||= Service =||
    191 || {{{/account/*}}} || repository2:5017 || omf-account-mgmt ||
    192 || {{{/scheduler/*}}} || am5:5016 || omf-scheduler ||
    193 || {{{/rfmatrix/*}}} || am5:5020 || omf-rfmatrix ||
    194 || {{{/status/*}}} || am5:5021 || omf-status ||
    195 || {{{/inventory/*}}} || am5:5012 || omf-newinventory (legacy) ||
    196 || {{{/user-stats/*}}} || repository2:5015 || omf-user-stats ||
    197 || {{{/mqtt/ws}}} || amqp.orbit-lab.org:15675/ws || RabbitMQ WebSocket ||
    198 
    199 == 7. Port Registry ==
    200 
    201 ||= Port =||= Service =||= Deployment =||
    202 || 5000 || omf-cmonitor || Console servers (18 hosts, per domain) ||
     129|| '''Port''' || '''Service''' || '''Deployment''' ||
     130|| 5000 || omf-cmonitor || Console servers (18 instances) ||
    203131|| 5001 || omf-rf-control || am1 ||
    204132|| 5002 || omf-rf-switch || am1 ||
    205133|| 5003 || omf-xy-table || am1 ||
    206134|| 5004 || omf-array-mgmt || am1 ||
    207 || 5010 || omf-pxe || repository2 ||
    208 || 5011 || omf-frisbee || repository2 ||
    209 || 5012 || omf-saveimage || repository2 ||
     135|| 5010 || omf-pxe || repository2, am-crf ||
     136|| 5011 || omf-frisbee || repository2, am-crf ||
     137|| 5012 || omf-saveimage || repository2, am-crf ||
    210138|| 5013 || omf-cmc || am5 ||
    211139|| 5015 || omf-user-stats || repository2 ||
    212140|| 5016 || omf-scheduler || am5 ||
    213141|| 5017 || omf-account-mgmt || repository2 ||
    214 || 5018 || omf-cosmos-cm || COSMOS Pis ||
    215 || 5019 || omf-auden || COSMOS Pis ||
     142|| 5018 || omf-cosmos-cm || Pi nodes ||
     143|| 5019 || omf-auden || Pi nodes ||
    216144|| 5020 || omf-rfmatrix || am5 ||
    217145|| 5021 || omf-status || am5 ||
    218 || 5054 || omf-agg-mgr-proxy || am1 ||
    219146
    220 '''Next available port: 5022'''
     147== Network Architecture ==
    221148
    222 == 8. Technology Stack ==
     149=== IP Ranges ===
     150|| '''Range''' || '''Site''' || '''Usage''' ||
     151|| 10.0.0.0/16 || NJ || Management infrastructure ||
     152|| 10.1.0.0/16 || NJ || Control network (ORBIT domains) ||
     153|| 10.50.0.0/16 || NJ || Server infrastructure ||
     154|| 10.250.0.0/16 || NJ || DHCP/DNS management ||
     155|| 10.64.0.0 - 10.127.0.0 || NYC || COSMOS networks ||
    223156
    224 === Backend ===
     157=== MQTT Broker ===
     158 * Server: amqp.orbit-lab.org (10.50.0.44), port 1883
     159 * Used by: omf-expctl (EC) ↔ omf-resctl (RC on PXE-booted nodes)
     160 * Topic format: omf/<fqdn-with-dashes>/{cmd,reply,status}
     161 * WebSocket: port 15675/ws (proxied through web1 for portal)
    225162
    226  * '''Ruby 3.2.x''' -- Primary language for all microservices
    227  * '''Sinatra''' (v4.x) -- Web framework (all services inherit from {{{OMFService}}} base class)
    228  * '''Puma''' -- Application server
    229  * '''ActiveRecord''' -- ORM for MySQL-backed services (scheduler, account-mgmt, user-stats)
    230  * '''Ox''' -- Fast XML parser/generator for OMF XML responses
    231  * '''Python 3.x / Flask''' -- omf-array-mgmt only
    232  * '''sinatra-param''' -- Request parameter validation via DSL
     163== Configuration Management ==
    233164
    234 === Data Stores ===
     165All services load configuration in this order (later overrides earlier):
     166 1. {{{default/<service>.yml}}} — built-in defaults (shipped in .deb)
     167 2. {{{/etc/omf-services/common.yml}}} — global settings (MySQL, LDAP)
     168 3. {{{/etc/omf-services/<service>.yml}}} — service-specific overrides
     169 4. {{{./config.yml}}} — development-only (not installed)
    235170
    236  * '''MySQL / MariaDB''' -- Persistence for scheduler, account-mgmt, user-stats, rfmatrix
    237  * '''OpenLDAP''' -- User/group directory (ldap1/ldap2, port 389)
    238  * '''RabbitMQ''' -- MQTT broker for node communication and XY table telemetry
     171== Shared Components ==
    239172
    240 === Frontend ===
     173=== Git Submodules ===
     174|| '''Submodule''' || '''Purpose''' ||
     175|| omf-common || Sinatra base class, DSL, XML helpers, config loading ||
     176|| omf-logging-db || ActiveRecord database modules (usage, imaging, login audit) ||
     177|| omf-ldap || LDAP client wrapper (used by omf-account-mgmt, omf-scheduler) ||
    241178
    242  * '''React 18''' -- cosmos-portal SPA
    243  * '''Vite''' -- Build tool
    244  * '''Tailwind CSS''' -- Styling
    245  * '''Apache''' -- Static file serving + reverse proxy on web1
     179=== PXE Images ===
     180|| '''Image''' || '''Kernel''' || '''Purpose''' ||
     181|| omf-5.8 || 6.18 LTS (Alpine 3.23) || Regular PCs + Dell servers ||
     182|| omf-5.9 || 5.15 LTS || SuperMicro/Thinkmate (kernel 6.18 crashes) ||
     183|| omf-5.10 || TBD || Hardware inventory scanning (with UHD) ||
    246184
    247 === Operations ===
    248 
    249  * '''Debian packaging''' via FPM ({{{make deb}}})
    250  * '''systemd''' service units (auto-enabled on package install)
    251  * '''Prometheus''' metrics at {{{/metrics}}} on all services
    252  * '''LibreNMS''' -- Network device monitoring (190 devices via SNMP)
    253  * '''Git submodules''' -- {{{omf-common}}} (shared framework), {{{omf-logging-db}}} (database helpers), {{{omf-ldap}}} (LDAP helpers)
    254 
    255 === Configuration ===
    256 
    257 Configuration is merged in order (later files override earlier):
    258 
    259  1. {{{default/config.yml}}} -- Built-in defaults (shipped with package)
    260  2. {{{/etc/omf-services/config.yml}}} -- Global settings
    261  3. {{{/etc/omf-services/<service>.yml}}} -- Service-specific (e.g., {{{cmonitor.yml}}})
    262  4. {{{./config.yml}}} -- Development override (not installed in production)
    263 
    264 === PXE Boot Images ===
    265 
    266  * '''omf-5.8''' (current) -- Alpine 3.23, kernel 6.18 LTS, Ruby 3.4.8, MQTT-based RC, 97MB initfs
    267  * '''omf-5.7''' (legacy) -- Alpine LTS 5.15, Ruby 3.1, XMPP-based RC, 41MB initfs
    268  * PXE images served from {{{root@repository2:/tftpboot/}}}
     185''Last updated: March 2026''