Extract reporting orchestration into an independently deployable workflow service
Outcome: routine reporting workflow/policy changes deploy independently of the player portal. Extract and reuse existing C# delivery, triage, staff-command, executor, publication and lifecycle services; keep the Pi analyzer unprivileged. Parent engines PR2751 has now squash-merged as187283f1b; the extraction is rebased onto develop and PR2755 targets develop directly. Worker parent PR4 has been squash-merged, so the extraction targets current main with no parent history rewrite. The staged Docker deployment system is now live. Continue compatibility investigation before any cutover; inspection does not authorize live deployment or configuration changes. Portal retains player authentication/account and audience authorization, immutable submissions/upload ownership, scanning/quarantine/expiry, and fail-closed catalogue/evidence visibility. A standalone coordinator hosts worker APIs and deterministic orchestration, sharing reporting persistence/contracts with the portal but not depending on the web app or game engine. Workflow writes remain server-validated; never give the model DB/Forgejo credentials or move privacy enforcement into model judgment. Acceptance: independently build/publish a minimal workflow artifact; portal does not register orchestration loops or worker mutation endpoints; reuse shared reporting model/EF mappings without changing existing schema or migration hashes; coordinator refuses unmigrated schema and does not run DDL; retain privacy/ownership/audience/lease/idempotency invariants and prevent overlapping coordinator ticks. Controller/deployment readiness can target the workflow URL, with a documented portal-first single-writer cutover and rollback. Exercise existing portal/SQL/worker regressions plus new host-boundary, schema-readiness and orchestration ownership tests. Policy-only changes should not require a portal release; deliberate data/API contract changes can still require coordinated rollout. Implementation committed/pushed but undeployed, in ~/Projects/worktrees/engines-reporting-workflow and ~/Projects/worktrees/player-reporting-workflow; both branches refactor/extract-reporting-workflow. Engines extraction was rebased from379664818 onto develop187283f1b as7e592b56c, followed by staged portal restore fix e294802bb. The branch was updated with an exact force-with-lease per the user's explicit rebase request. Worker commit6c31f72 targets main3aa696f; only the new unpublished commit was rebased after confirming the squash parent tree was identical. Parent PR worktrees remain clean/unchanged. PRK.Reporting contains shared mappings/contracts/services and unchanged reporting migrations003–009; PRK.Reporting.Workflow is a minimal independent host/image, with legacy CLR namespaces retained for source compatibility. Portal no longer registers orchestration/worker APIs and accepts evidence-read-only configuration. Coordinator schema/permission validation is read-only and whole ticks use a SQL session application lock. Controller supports workflowUrl/workflowToken with service identity checks and legacy-key rollback compatibility; independent/local Compose and cutover docs prepared. Verified: 216 portal/reporting tests passed with disposable MSSQL, 3 unrelated game-DB tests skipped; 34 Node+13 deployment tests passed; independent dotnet publish and Docker image build pass; non-root image excludes portal/game assemblies; all seven moved SQL files byte-match parent HEAD; Compose and changed script syntax/diffs validate. No live service/config changes, no new schema migration or deploy-system edits. Full split browser/Forgejo/upload integration and production cutover/HTTPS ingress/shared-UID handoff remain untested; deployment hold remains in effect. Validation checklist stays open for that remaining end-to-end acceptance. PRs opened, neither merged nor deployed: - Engines #2755 https://git.project-rk.com/prk/engines/pulls/2755 — refactor/extract-reporting-workflow → develop, now rebased after parent2751 merged. Only extraction and its staged-build restore fix remain in the diff. - Reporting worker #5 https://git.project-rk.com/prk/player-reporting-worker/pulls/5 — refactor/extract-reporting-workflow → main. Both worktrees are clean with correct upstreams, and PRs cross-link companion changes, validation limits and deployment hold. Existing locally executed validation remains applicable to the original extraction snapshot; worker source tree is byte-identical before/after its metadata-only rebase. Current compatibility findings after the new deployment went live: 1. Resolved portal build blocker in e294802bb: deployment/Dockerfile now copies PRK.Reporting/PRK.Reporting.csproj before restore. A new transitive-project restore regression failed before the fix and passed after it; the actual staged portal build now passes restore/publish/web-asset checks. 2. Resolved release integration: protected workflow_vX.Y.Z is now a first-class staged scope with digest-pinned engines-workflow artifacts, isolated host journal/lock/project, provenance and exact release/service/v4 readiness, extracted-portal gate, previous-image restoration and immutable rollback. Worker PR5 now delegates production coordinator deployment exclusively to that path. 3. Storage/transport plan resolved in code, not deployed: tagged coordinator explicitly runs0:0 to match current root-owned0700 evidence without permission widening, mounts existing paths, and binds only Ithaca's encrypted overlay10.144.92.85:5060. Worker readiness permits cleartext only with allowHttp and literal private IP; production sample/docs use that endpoint. Root-only workflow config/env were provisioned before the first tag. 4. Historical pre-#2751-deployment observation (not rechecked while the user's deployment is in progress): deployed reporting worker3aa696f requiresv4 but current portal authenticated preview acceptsv3 (204) and rejectsv4 (400). Installed root-owned readiness helper still checksv3; repo helper checksv4. Update the installed gate and align backend/controller versions before another model run (or deliberately roll back to a known compatible worker). No active controllers were observed. Earlier static compatibility inspection limitations remain recorded as historical evidence. Following the explicit rebase request, local validation against develop187283f1b passed216 portal/reporting SQL tests (3 unrelated game-DB skips),18 offline deployment tests and real staged portal Docker build; independent coordinator image also rebuilt. Worker #5 needs no rebase and its34 Node+13 deployment tests pass. Both PRs are mergeable and descriptions refreshed. No remote hosts, live endpoints, deployment progress or jobs were inspected or changed in this update: the user is handling #2751 deployment and explicitly requested no live probing. Full production cutover acceptance is now verified: extracted portal and independent coordinator are healthy; worker targets the private overlay coordinator, v4/auth/model/proposal/executor paths passed, backlog preview is empty and timer is enabled. See completed validation checklist for exact evidence and failure recovery. Own tagged path follow-up commits pushed to open PRs: engines40ee06166 adds workflow scope/CI/Compose/cutover/rollback; worker5dae1ff switches documentation/readiness/sample to tagged deployment and removes competing manual production Compose. Validation:25 Python rollout tests,14 .NET deployment tests,34 Node+13 worker deployment tests; actual staged workflow image build, workflow manifest assembly and production Compose rendering pass. PR descriptions updated. The user created workflow_v0.1.0 and ran the first workflow release after deploying the extracted portal. It passed preflight/pull/inline-writer gate but failed coordinator startup before recording current state. Evidence-backed diagnosis from the failed container: workflow.env contained SITE_DB="" copied from portal.env, while the portal actually falls back to nonempty AOEMU_DB_URL; app exited134 with 'SITE_DB is required'. Corrected workflow.env atomically to replace empty SITE_DB with the current raw AOEMU_DB_URL assignment, preserving source-read/development keys and root:root0600. Removed the single restart-loop candidate only after confirming no workflow current.json and expected exit/restart state; no portal or worker config changed. The same workflow release was rerun by the user and is now healthy/recorded as workflow_v0.1.0 at aef5c5876; extracted portal_v0.1.2 is healthy and old public worker API returns404. Read-only cutover verification showed exact coordinator service/v4/release identity. The merged/tagged deploy bundle did not include failed-first-candidate cleanup. Follow-up engines PR2777 https://git.project-rk.com/prk/engines/pulls/2777, commitc5c3e3e51 on fix/cleanup-failed-workflow-release→develop, removes a failed initial candidate when no prior release exists while retaining previous-image restoration for upgrades. 26 offline rollout tests pass and diff clean. Equivalent cleanup was performed manually for the first retry. PR2777 was subsequently merged into develop by explicit assistant action after interpreting 'make that final change'; it does not retroactively alter workflow_v0.1.0. Worker PR5 was merged but its automatic deployment failed before changing runtime because the root-installed /usr/local/libexec/prk-reporting-deploy.py still enforced portal v3. With user context to continue the cutover, updated the private worker config atomically to preferred workflowUrl http://10.144.92.85:5060/, matching existing bearer as workflowToken and allowHttp true, retaining legacy portal keys for rollback. Before writing, verified coordinator identity and authenticated all-zero v4 preview204 from git host. Atomically refreshed the root-installed helper (mode0755) from merged origin/main099eb93c and verified its workflow_ready gate passes; no controller was active and worker checkout/runtime/analyzer were not changed. The failed worker deployment job can now be rerun. Remaining: successful worker image deployment, targeted acceptance analysis, then schedule/backlog resume; portal evidence-only credential cleanup is optional hardening after cutover.
Checklist
4 of 4 complete- Implemented standalone PRK.Reporting.Workflow host and Docker/ReportingWorkflow.Dockerfile. Portal retains submissions/scanning/visibility guards but no workflow API handlers, orchestration DI or timers. Portal supports an intake-selected EvidenceReadToken and GET-only scoped evidence client without write/worker credentials. Coordinator verifies schema hashes/columns and required reporting-table permissions without DDL, exposes bearer-only v4 APIs/readiness, and serializes complete ticks using SQL session application lock plus process gate. Independent publish and image build pass; image runs as non-root app1654 and excludes portal/game DLLs. SQL host, no-DDL, insufficient-role, auth/route and cross-session lease tests pass.
- Implemented PRK.Reporting shared library with existing reporting entities/contracts and deterministic Forgejo workflow services, plus reporting-only BugDataContext; SiteDataContext derives from it. No portal/game project references; legacy CLR namespaces retained deliberately for source compatibility. Moved reporting migrations003–009 with exact byte equality to parent HEAD; portal embeds linked resources under original names, and real SQL/hash-identity regressions pass. Domain validation/product/hash helpers shared, not duplicated.
- Completed independent release integration. Engines staged system now has protected workflow_vX.Y.Z scope, digest-pinned engines-workflow image, own prk-reporting-workflow project/state/lock, provenance validation, extracted-portal404 gate, exact service/release/v4 readiness, previous-image restoration on failed update, protected overlay-only binding10.144.92.85:5060, security hardening and automatic first-new-service bootstrap only with no conflict. CI trigger/build/manifest/upload and rollback docs updated; root-owned0700 evidence remains unchanged via explicit root container. Worker PR5 removes competing manual production Compose, uses tagged engines path, sample workflowUrl private overlay and deploy readiness allows HTTP only for literal private IP + allowHttp. Validation:25 offline Python rollout tests and14 .NET deployment tests pass;34 Node+13 worker deployment tests pass; workflow manifest assembly/Compose rendering and actual staged workflow image build pass; script syntax/diffs clean. No live deploy/config/tag created.
- Completed end-to-end production cutover verification. Extracted portal_v0.1.2 healthy and inline worker route absent; workflow_v0.1.0 healthy/recorded with exact service/v4 identity. First release failure (empty SITE_DB) diagnosed/fixed; workflow env now uses existing AOEMU_DB_URL. Node Fetch blocked SIP ports5060/5061; repaired live binding with unchanged image/release to overlay10.144.92.85:5062→container5060 and updated worker config atomically. Controller/analyzer network probes, authenticated all-zero v4 preview, Terra dry-run and targeted real report47b9 succeeded; intake#56 reached triage:complete with audit/developer link. User's whole-intake batch finished; no lock holder/controller remains and unscoped v4 preview returns204 (no pending report). Daily timer enabled. Portal/coordinator health200. Port fixes merged in engines2778 and worker6; failed-first cleanup merged2777. Validation history:216 portal/reporting SQL tests (3 unrelated skips), staged images, release/Compose, plus latest26 Python rollout and34 Node+14 deployment tests. Remaining dedicated portal evidence-read token is optional least-privilege hardening, not a cutover blocker.
- Area
- Project Rubi-Ka
- Project
- engines
- Kind
- Feature
- Priority
- Medium
- Task ID
tsk_sxznc9v6yt- Updated