Document 01 — Validation Master Plan (VMP)
| Document ID | Version | Effective Date (UTC) | System Name | Regulatory Scope | Status |
|---|---|---|---|---|---|
VMP-GXPHISTORIAN-2026-001 | 1.0 | 2026-10-17T21:20:51Z | GxPHistorian GxP Industrial Historian (ClickHouse + PostgreSQL Part 11 Control Plane) | 21 CFR Part 11, EU GMP Annex 11, ISPE GAMP 5 (2nd Ed.), FDA CSA | APPROVED |
1. Purpose & Regulatory Scope
This Validation Master Plan (VMP) establishes the Computerized System Validation (CSV) and Computer Software Assurance (CSA) governance framework for the GxPHistorian GxP Industrial Historian deployed at BioCorp Fremont Biomanufacturing Operations. The platform replaces legacy proprietary OSIsoft PI System components with a 4-Pillar open-source architecture engineered for 21 CFR Part 11, EU EudraLex Volume 4 Annex 11, and ALCOA+ data integrity compliance.
2. System Classification (ISPE GAMP 5 Second Edition)
| Subsystem Pillar | Component | GAMP 5 Category | Validation Strategy |
|---|---|---|---|
| Pillar 1: Edge Collector | Telegraf 1.32 + Disk WAL + Python Deadband Worker | Category 4 / Category 5 | Automated IQ + Scripted OQ on Stage 1 Exception/Deadband logic & WAL persistence |
| Pillar 2: Telemetry Archive | ClickHouse Server 24.8 LTS (MergeTree / AggregatingMergeTree) | Category 4 | Automated IQ on lossless codecs (DoubleDelta, Gorilla) + Scripted OQ on quality >= 192 filtering, LTTB, and interpolation |
| Pillar 3: Control Plane | PostgreSQL 16 + FastAPI (historian-api) | Category 4 / Category 5 | Scripted OQ on PL/pgSQL Part 11 WORM audit triggers, SHA-256 hash chaining, and §11.200 electronic signatures |
| Pillar 4: Operator HMI | Grafana OSS 11.2 + Excel DataLink UDF/Power Query | Category 4 | Automated IQ on provisioned datasources/dashboards + OQ on DataLink CSV/JSON parity |
3. Roles & Responsibilities
| Role | Designated Personnel | GxP Responsibilities |
|---|---|---|
| System Owner | Director of Manufacturing IT/OT | Accountable for system availability, access control, backup/restore drills, and GitOps change control. |
| Process Owner | Head of Upstream & Downstream MSAT | Defines User Requirements (URS), engineering units, alarm limits (low_limit, high_limit), and batch Event Frame templates. |
| Quality Assurance (QA) | Lead CSV / Part 11 Qualified Person (qa_approver_01) | Approves VMP, URS, Risk Assessment, executes/reviews IQ/OQ evidence, and signs batch/excursion Event Frames. |
| Validation Engineering | Senior CSV Automation Engineer | Authors and maintains executable pytest IQ/OQ qualification suites and generates the cryptographic validation binder. |
4. Validation Lifecycle (IQ → OQ → PQ)
- Installation Qualification (IQ): Automated verification (
test_01_iq_infrastructure.py) of container health, non-default credentials, PostgreSQL extensions (ltree,btree_gist,pgcrypto), ClickHouse lossless compression codecs, and dictionary bridging. - Operational Qualification (OQ): Scripted positive and adversarial negative testing (
test_02_oq_part11_compliance.py&test_03_oq_historian_core.py) verifying 21 CFR Part 11 WORM immutability, SHA-256 hash chain verification, electronic signature credential challenges, bad quality (quality < 192) rejection, linear vs. LOCF step interpolation, and LTTB peak preservation. - Performance Qualification (PQ): Continuous multi-protocol plant telemetry ingestion across 5 GxP tags (
101,102,103,301,401), live Event Frame excursion capture, and Apache Parquet lakehouse exports.
5. Backup, Restore, Disaster Recovery & GitOps Change Control SOP
- Database & Telemetry Backups: PostgreSQL
pi_control_plane(includingpart11_audit_log) undergoes daily logicalpg_dumpand continuous WAL archiving to immutable object storage. ClickHousepi_historianpartitions (PARTITION BY toYYYYMM(ts)) are exported to Snappy-compressed Apache Parquet files and backed up via native ClickHouseBACKUP TABLEsnapshots. - GitOps Change Control (
SOP-IT-CHG-014): All infrastructure definitions (docker-compose.yml), database schemas (init-postgres/,init-clickhouse/), and API code (app/main.py) are version-controlled in Git. Any change requires a pull request, automated execution of the 15-test IQ/OQpytestsuite (100%pass gate), and QA electronic approval prior to production deployment.
Document 02 — User Requirements Specification (URS)
Document ID: URS-GXPHISTORIAN-2026-002 | Generated (UTC): 2026-10-17T21:20:51Z | Total Requirements: 17
1. Formal User Requirements Catalog (URS-001 to URS-017)
| URS ID | Requirement Title & Operational Statement | GxP Risk | ALCOA+ Attributes | Regulation Ref | Acceptance Criteria (FRS) |
|---|---|---|---|---|---|
| URS-001 | Multi-Service Container Infrastructure & Non-Default GxP Security Credentials The system shall deploy isolated PostgreSQL Control Plane, ClickHouse Telemetry Archive, Edge Collector, FastAPI Control Plane, and Grafana HMI services using non-default authenticated credentials. | Medium | Attributable, Contemporaneous | 21 CFR §11.10(d), EU GMP Annex 11 §12 | FRS-INFRA-001: All database, API, and HMI endpoints enforce non-default credentials configured via environment variables and pass readiness health checks. |
| URS-002 | PostgreSQL 16 Control Plane Extensions & Core GxP Schema Qualification The PostgreSQL Control Plane shall enable ltree, btree_gist, and pgcrypto extensions and provision tables for asset_templates, assets, tags, event_frames, users, and part11_audit_log. | Medium | Legible, Enduring | 21 CFR §11.10(a), GAMP 5 Appendix M4 | FRS-PG-002: PostgreSQL 16 initializes ltree, btree_gist, and pgcrypto extensions, GiST hierarchy indexes, and all 6 required GxP tables. |
| URS-003 | ClickHouse Lossless Time-Series Compression Codecs Verification The ClickHouse Telemetry Archive shall store high-frequency sensor data using lossless Delta, DoubleDelta, Gorilla, and T64 codecs layered with ZSTD(1). | Low | Original, Accurate, Enduring | 21 CFR §11.10(b), EU GMP Annex 11 §7.1 | FRS-CH-003: Table pi_historian.sensor_data enforces CODEC(Delta, ZSTD(1)) on tag_id, CODEC(DoubleDelta, ZSTD(1)) on ts, CODEC(Gorilla, ZSTD(1)) on val, and CODEC(T64, ZSTD(1)) on quality and status_code. |
| URS-004 | ISA-95 ltree Asset Hierarchy & ClickHouse In-Memory Dictionary Synchronization The system shall synchronize ISA-95 hierarchical asset paths and tag metadata from PostgreSQL into ClickHouse via an in-memory FLAT dictionary. | Medium | Legible, Consistent | ISA-95, 21 CFR §11.10(a) | FRS-DICT-004: ClickHouse dictionary pi_historian.pg_asset_tags resolves all seeded GxP tags and their ISA-95 asset_path strings from PostgreSQL v_clickhouse_tag_dict. |
| URS-005 | Stage 1 Edge Exception/Deadband Filtering & Disk WAL Store-and-Forward Buffer The Edge Collector shall perform Stage 1 Exception/Deadband filtering (ExcDev and ExcMax) and persist outbound batches in a disk-backed Write-Ahead Log (WAL) buffer. | High | Complete, Original, Available | ALCOA+ (Complete, Original), EU GMP Annex 11 §7 | FRS-EDGE-005: Telegraf edge-collector configures buffer_strategy='disk' at /var/lib/telegraf/buffer and applies ExcDev/ExcMax deadband filtering accurately. |
| URS-006 | 21 CFR Part 11 Mandatory Attribution Context Enforcement All INSERT, UPDATE, and DELETE operations on GxP configuration and batch Event Frame tables shall require authenticated actor_id and change_reason context or be rejected. | High | Attributable, Contemporaneous | 21 CFR §11.10(e), ALCOA+ (Attributable) | FRS-P11-006: PL/pgSQL trigger fn_enforce_part11_audit() raises '21 CFR Part 11 Violation: app.actor_id and app.change_reason are mandatory' when session context is missing. |
| URS-007 | 21 CFR Part 11 §11.10(e) WORM Audit Trail Immutability The computer-generated, time-stamped audit trail (part11_audit_log) shall be strictly append-only (Write-Once-Read-Many) and block UPDATE, DELETE, and TRUNCATE operations. | High | Original, Enduring, Complete | 21 CFR §11.10(e), EU GMP Annex 11 §9 | FRS-P11-007: PL/pgSQL trigger fn_prevent_audit_tampering() unconditionally raises '21 CFR Part 11 WORM Violation: Audit trail records are immutable' on UPDATE, DELETE, or TRUNCATE. |
| URS-008 | 21 CFR Part 11 §11.200 Electronic Signature Rejection of Unauthorized Credentials The system shall reject electronic signature attempts with invalid user IDs, wrong passwords, or missing change reasons. | High | Attributable, Accurate | 21 CFR §11.10(d), §11.200(a) | FRS-P11-008: POST /api/v1/event-frames/{event_id}/sign verifies password against pgcrypto bcrypt hash and returns HTTP 401 on invalid credentials without altering the record. |
| URS-009 | 21 CFR Part 11 §11.50 & §11.70 Electronic Signature Manifestation & HMAC Binding Signed Event Frames shall record the signer's user ID, timestamp, signature meaning, and cryptographic HMAC-SHA256 signature hash linked to the audit trail. | High | Attributable, Legible, Enduring | 21 CFR §11.50, §11.70, §11.200 | FRS-P11-009: POST /api/v1/event-frames/{event_id}/sign authenticates qa_approver_01, binds HMAC-SHA256 over (event_id, user_id, signature_meaning, signed_at), and logs to part11_audit_log. |
| URS-010 | 21 CFR Part 11 Cryptographic SHA-256 Audit Hash-Chain Verification Every audit trail entry shall chain the previous record's SHA-256 hash from GENESIS_0000000000000000, and the API shall verify end-to-end cryptographic integrity. | High | Original, Accurate, Enduring | 21 CFR §11.10(e), ALCOA+ (Enduring, Accurate) | FRS-P11-010: GET /api/v1/audit-trail/verify recomputes the SHA-256 hash chain across all part11_audit_log records and confirms chain_valid == true. |
| URS-011 | Multi-Protocol Telemetry Ingestion (OPC UA, BACnet/IP, Siemens S7, Legacy OPC DA) The historian shall ingest and normalize real-time telemetry across OPC UA, BACnet/IP, Siemens S7, and legacy Windows OPC DA bridges into ClickHouse. | Medium | Contemporaneous, Original | 21 CFR §211.68, ALCOA+ (Contemporaneous) | FRS-HIST-011: All 5 multi-protocol tags (101, 102, 103, 301, 401) plus HTTP /ingest batches are archived in pi_historian.sensor_data with UTC millisecond timestamps. |
| URS-012 | Bad OPC Signal Quality (quality < 192) Exclusion from GMP Rollups Telemetry points flagged with Bad (0) or Uncertain (64) OPC quality codes shall be stored in raw history for diagnostics but strictly excluded from 1-minute GMP rollups. | High | Accurate, Consistent | 21 CFR §211.188, ALCOA+ (Accurate) | FRS-HIST-012: Materialized view pi_historian.mv_sensor_rollup_1m filters WHERE quality >= 192, excluding injected quality=0 fault spikes (e.g., val=9999.0) from min/max/avg rollups. |
| URS-013 | PI DataLink Parity: Linear Interpolation vs. Discrete Step (LOCF) Sampling The sampled data endpoint shall linearly interpolate continuous analog tags (is_step_tag=False) and apply Last Observation Carried Forward (LOCF) to discrete step tags (is_step_tag=True). | High | Accurate, Complete | 21 CFR §11.10(b), GAMP 5 | FRS-DL-013: GET /api/v1/datalink/sampled computes exact linear midpoints for continuous tags and strict integer LOCF states for step tags without fractional artifacts. |
| URS-014 | PI Vision PlotValues Parity: LTTB Visual Downsampling Peak Preservation Visual trend downsampling shall use ClickHouse largestTriangleThreeBuckets (LTTB) to preserve exact local excursion peaks and valleys. | Medium | Accurate, Legible | 21 CFR §11.10(b) | FRS-LTTB-014: GET /api/v1/trends/plot-values preserves the exact timestamp and magnitude of a synthetic spike when downsampling high-density telemetry. |
| URS-015 | ISA-88 Event Frame Excursion Capture & Zero-ETL Apache Parquet Lakehouse Export The system shall automatically detect parameter excursions into Event Frames with summary KPIs and export enriched telemetry to Apache Parquet files with 100% fidelity. | High | Available, Enduring, Complete | 21 CFR §11.10(b), ISA-88 | FRS-LAKE-015: POST /api/v1/event-frames/evaluate captures excursion metrics in PostgreSQL and POST /api/v1/lakehouse/export-parquet writes a valid Snappy-compressed Parquet file verified via pyarrow. |
| URS-016 | Pydantic AI Advisory Deviation Investigator & Read-Only Model Context Protocol (gxphistorian-mcp) The AI Co-Pilot and MCP Server shall execute all time-series and audit queries deterministically in ClickHouse (WHERE quality >= 192, LINEAR vs LOCF_STEP) and PostgreSQL while loading specialist forensic capabilities on demand. | High | Accurate, Consistent, Available | ISPE GAMP 5 Appendix D11, 21 CFR §11.10(e) | FRS-AI-016: POST /api/v1/ai/event-frames/{event_id}/investigate and GET /api/v1/ai/mcp/manifest return a validated DeviationInvestigationBrief with part11-forensics capability and unbroken SHA-256 chain verification. |
| URS-017 | Pydantic AI Human-in-the-Loop 21 CFR Part 11 Deferred Tool Approval Gate Any AI-proposed GxP tag provisioning shall validate engineering bounds via args_validator, pause execution with ApprovalRequired (DeferredToolRequests), and require human approval with X-Actor-Id and X-Change-Reason headers. | High | Attributable, Contemporaneous, Original | 21 CFR §11.10(e), EU GMP Annex 11 §11, FDA CSA | FRS-AI-017: POST /api/v1/ai/copilot/propose-tag pauses with approval_required when approved=False, rejects missing Part 11 headers with HTTP 400, and records WORM audit logs upon approved=True. |
Document 03 — Functional & Design Specification (FDS)
Document ID: FDS-GXPHISTORIAN-2026-003 | Effective Date (UTC): 2026-10-17T21:20:51Z
1. 4-Pillar System Architecture
- Pillar 1 — Multi-Protocol Edge Collector (
edge-collector): - Built on
telegraf:1.32-alpinewithbuffer_strategy = "disk"at/var/lib/telegraf/buffer(metric_buffer_limit = 1000000). - Exposes
[[inputs.http_listener_v2]]on:8080/ingestfor Windows localhost COM OPC DA forwarders (opcda_windows_forwarder.py) and asyncbacpypes3BACnet/IP workers (bacnet_cov_worker.py). - Executes
multi_protocol_simulator.pyvia[[inputs.execd]]implementing Stage 1 Exception/Deadband filtering (ExcDevandExcMax). - Pillar 2 — Immutable Telemetry Archive (
clickhouse-archive): clickhouse/clickhouse-server:24.8-alpinedatabasepi_historian.- Table
pi_historian.sensor_data(ENGINE = MergeTree() PARTITION BY toYYYYMM(ts) ORDER BY (tag_id, ts)): tag_id UInt32 CODEC(Delta, ZSTD(1))ts DateTime64(3, 'UTC') CODEC(DoubleDelta, ZSTD(1))val Float64 CODEC(Gorilla, ZSTD(1))str_val LowCardinality(String) DEFAULT '' CODEC(ZSTD(1))quality UInt8 DEFAULT 192 CODEC(T64, ZSTD(1))status_code UInt16 DEFAULT 0 CODEC(T64, ZSTD(1))- Real-time 1-minute rollup table
pi_historian.sensor_rollup_1m(AggregatingMergeTree()) populated bymv_sensor_rollup_1m(WHERE quality >= 192). - In-memory dictionary
pi_historian.pg_asset_tags(LAYOUT(FLAT()) LIFETIME(MIN 5 MAX 15)) and external tablepi_historian.pg_event_frames. - Pillar 3 — PostgreSQL 16 Part 11 Control Plane, FastAPI & Pydantic AI / MCP (
postgres-control+historian-api): - PostgreSQL 16 extensions:
ltree,btree_gist,pgcrypto. - Tables:
asset_templates,assets(GiST index onasset_path),tags,event_frames,users, andpart11_audit_log. - Trigger
fn_enforce_part11_audit()enforcescurrent_setting('app.actor_id', true)andcurrent_setting('app.change_reason', true)and computesrow_hash = encode(digest(prev_hash || recorded_at::text || actor_id || operation || coalesce(new_payload::text, ''), 'sha256'), 'hex'). - Trigger
fn_prevent_audit_tampering()blocksUPDATE,DELETE, andTRUNCATEonpart11_audit_log. - Pydantic AI Co-Pilot (
gxp_ai_agent.py) &gxphistorian-mcp(mcp_server.py) provide read-only deterministic tools, progressive disclosure capabilities (part11-forensics,brownfield-onboarding), and human-in-the-loopApprovalRequired(DeferredToolRequests) Part 11 change control. - Pillar 4 — Operator HMI & Excel DataLink (
grafana-vision+excel-datalink-client): - Grafana OSS 11.2.0 pre-provisioned with
clickhouse_coreandpostgres_controldatasources and the asset-relative dashboardGxPHistorian Vision — GxP Bioreactor & Cleanroom HMI. - Excel DataLink UDF module (
gxphistorian_datalink_udf.py) and native Power Query CSV endpoint (/api/v1/datalink/sampled?format=csv).
Document 04 — FDA CSA Functional Risk Assessment (FRA)
Document ID: FRA-GXPHISTORIAN-2026-004 | Generated (UTC): 2026-10-17T21:20:51Z | Methodology: ISPE GAMP 5 & FDA Computer Software Assurance (CSA)
1. Functional Risk Assessment Matrix
| URS ID | Failure Mode / GxP Hazard | FMEA Score | CSA Risk Class | Required Testing Rigor | Technical Control & Verification Reference |
|---|---|---|---|---|---|
| URS-001 | Unauthorized network access using default vendor credentials | S=3, O=1, D=1 | Medium | Automated IQ Verification | Non-default secrets in .env verified by IQ test_iq_01 |
| URS-002 | Missing database extensions or corrupted control plane schema | S=3, O=1, D=1 | Medium | Automated IQ Verification | Automated DDL init script + IQ schema inspection test_iq_02 |
| URS-003 | Uncompressed storage exhaustion or lossy float precision loss | S=2, O=1, D=1 | Low | Automated IQ Verification | Lossless Gorilla/DoubleDelta codecs verified via system.columns in test_iq_03 |
| URS-004 | Stale or unmapped ISA-95 asset path in analytical queries | S=3, O=2, D=1 | Medium | Automated IQ Verification | ClickHouse FLAT dictionary with 5-15s refresh verified in test_iq_04 |
| URS-005 | Telemetry loss during temporary ClickHouse or network outage | S=4, O=2, D=1 | High | Scripted OQ / IQ Verification | Telegraf disk WAL buffer (/var/lib/telegraf/buffer) + ExcDev/ExcMax verified in test_iq_05 |
| URS-006 | Unattributed modification of GxP tag limits or Event Frames | S=5, O=2, D=1 | High | Scripted OQ (Negative & Positive Testing) | PL/pgSQL fn_enforce_part11_audit() blocks mutations lacking actor_id/change_reason (test_oq_01) |
| URS-007 | Malicious DBA or superuser deletion/alteration of audit trail | S=5, O=1, D=1 | High | Scripted OQ (Adversarial Tamper Challenge) | BEFORE UPDATE/DELETE/TRUNCATE WORM trigger fn_prevent_audit_tampering() (test_oq_02) |
| URS-008 | Spoofed or unauthorized electronic signature on Batch Event Frame | S=5, O=2, D=1 | High | Scripted OQ (Negative Security Testing) | pgcrypto bcrypt password verification returns HTTP 401 on invalid password (test_oq_03) |
| URS-009 | Detached or repudiated electronic signature on batch release | S=5, O=1, D=1 | High | Scripted OQ | HMAC-SHA256 signature binding over (event_id, user_id, meaning, signed_at) (test_oq_04) |
| URS-010 | Undetected offline tampering of historical audit trail rows | S=5, O=1, D=1 | High | Scripted OQ | Continuous SHA-256 hash chain from GENESIS_0000000000000000 verified via API (test_oq_05) |
| URS-011 | Protocol silo dropping OPC DA, BACnet/IP, S7, or OPC UA tags | S=3, O=2, D=1 | Medium | Automated OQ Verification | Unified multi-protocol collector verified across tags 101, 102, 103, 301, 401 (test_oq_06) |
| URS-012 | Broken sensor spike (quality=0, val=9999.0) corrupting batch CQA averages | S=5, O=3, D=1 | High | Scripted OQ (Fault Injection Challenge) | Mandatory WHERE quality >= 192 filter in mv_sensor_rollup_1m verified by fault injection (test_oq_07) |
| URS-013 | Linear interpolation creating invalid fractional states (1.5) on discrete valve/agitator tags | S=4, O=2, D=1 | High | Scripted OQ (Mathematical Boundary Verification) | Metadata-driven LOCF step vs. LINEAR interpolation in /api/v1/datalink/sampled (test_oq_08) |
| URS-014 | Visual trend downsampling hiding brief critical thermal excursion spikes | S=4, O=2, D=1 | Medium | Scripted OQ | ClickHouse largestTriangleThreeBuckets(LTTB) preserves exact local peaks (test_oq_09) |
| URS-015 | Corrupted or incomplete batch telemetry export to enterprise data lakehouse | S=4, O=1, D=1 | High | Scripted OQ | PyArrow Snappy Parquet export verified for 100% row/column fidelity (test_oq_10) |
| URS-016 | AI agent hallucinating time-series statistics or ignoring OPC quality < 192 | S=4, O=2, D=1 | High | Scripted OQ (Deterministic FunctionModel & TestModel Qualification) | All telemetry math executed deterministically in ClickHouse via Pydantic AI tools & gxphistorian-mcp (test_oq_11) |
| URS-017 | Autonomous AI agent mutating GxP tag catalog without human Part 11 attribution | S=5, O=2, D=1 | High | Scripted OQ (Negative & Positive Human-in-the-Loop Gate Testing) | Pydantic AI ApprovalRequired (DeferredToolRequests) + mandatory X-Actor-Id/X-Change-Reason gate (test_oq_12) |
Document 05 — SOUP / Open-Source Software (OSS) Vendor Qualification Dossier
Document ID: SOUP-GXPHISTORIAN-2026-005 | Generated (UTC): 2026-10-17T21:20:51Z | Standard: GAMP 5 Category 3/4 & ISO/IEC 62304 SOUP Qualification
1. Qualified Open-Source Software & Live Container SHA-256 Digest Matrix
| Service Container | Software Component | License | Corporate / Foundation Backing | GAMP Category | Live Container Image @sha256 Digest | Supply-Chain & GxP Controls |
|---|---|---|---|---|---|---|
clickhouse-archive | ClickHouse Server 24.8 LTS | Apache-2.0 | ClickHouse, Inc. | Category 4 (Configured Time-Series Archive) | sha256:b002e56ed5c16e224c312527f6fcba7e77216fec5d7a88a7828f59efc614feb5 | Alpine 3.20 minimal base, pinned @sha256 digest, read-only init mounts, non-default auth |
postgres-control | PostgreSQL 16 (ltree + pgcrypto) | PostgreSQL License (OSI Approved) | PostgreSQL Global Development Group | Category 4 (Configured Relational Control Plane) | sha256:cf78e76683b9ca8c5733cbbdce6c9262b45b6767934dd0a95e671f9a0fc20685 | Alpine minimal image, cryptographic pgcrypto SHA-256 & bcrypt triggers, WORM enforcement |
edge-collector | InfluxData Telegraf 1.32 + Python 3.12 Execd Worker | MIT License / PSF License | InfluxData / Python Software Foundation | Category 4 / Category 5 (Custom Deadband Script) | sha256:bd7b0253571b2b905ee37ee3529a8fe2de0ebfc669975508d3eb7482723ab2f2 | Disk-backed WAL (/var/lib/telegraf/buffer), local localhost COM/HTTP isolation |
historian-api | FastAPI 0.115 / Uvicorn / PyArrow 17 / Psycopg 3 | MIT / Apache-2.0 / LGPL-3.0 | FastAPI / Apache Software Foundation | Category 5 (Custom GxP Application Layer) | sha256:71c7021194aad3b77d38e072fec3bf2a842f46924337ddf6923fe3359e1ccc26 | Pinned requirements.txt versions, mandatory Part 11 header enforcement, 100% OQ test coverage |
grafana-vision | Grafana OSS 11.2.0 + ClickHouse Datasource Plugin | AGPL-3.0 / Apache-2.0 | Grafana Labs | Category 4 (Configured HMI Visualization) | sha256:408afb9726de5122b00a2576763a8a57a3c86d5b0eff5305bc994ceb3eb96c3f | Read-only provisioned datasources, non-default admin credentials, zero direct DB write privileges |
Document 06 — Executed IQ/OQ Protocol & Cryptographic Evidence Report
Document ID: EPR-GXPHISTORIAN-2026-006 | Session Start (UTC): 2026-10-17T21:20:39.505782+00:00 | Session End (UTC): 2026-10-17T21:20:41.678732+00:00 | Verdict: PASS (17/17 Passed, 100.0%)
1. Executed Automated IQ/OQ Test Results (oq_execution_results.json)
| Test ID | Pytest Node ID | Start (UTC) | Duration (s) | Executed Assertions & Evidence Summary | Verdict |
|---|---|---|---|---|---|
| TC-01 | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_01_services_and_non_default_credentials | 2026-10-17T21:20:39.970921+00:00 | 0.0404 | Verified PostgreSQL, ClickHouse, FastAPI, and Grafana health with non-default GxP secrets. | PASS |
| TC-02 | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_02_postgres_extensions_and_schema | 2026-10-17T21:20:40.021429+00:00 | 0.0084 | Verified PostgreSQL 16 extensions (btree_gist, ltree, pgcrypto), 6 GxP tables, and seeded Part 11 users. | PASS |
| TC-03 | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_03_clickhouse_compression_codecs | 2026-10-17T21:20:40.039575+00:00 | 0.003 | Verified ClickHouse pi_historian.sensor_data column codecs (Delta, DoubleDelta, Gorilla, T64 + ZSTD(1)). | PASS |
| TC-04 | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_04_clickhouse_postgres_dictionary_sync | 2026-10-17T21:20:40.050924+00:00 | 0.0035 | Verified all 5 seeded GxP tags and ISA-95 ltree paths resolve in ClickHouse pg_asset_tags dictionary. | PASS |
| TC-05 | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_05_edge_collector_wal_and_deadband | 2026-10-17T21:20:40.055345+00:00 | 0.0011 | Verified disk-backed WAL buffer configuration and Stage 1 ExceptionDeadbandFilter suppression/emission rules. | PASS |
| TC-06 | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_01_reject_mutations_without_actor_or_reason | 2026-10-17T21:20:40.069923+00:00 | 0.0144 | Confirmed both PostgreSQL PL/pgSQL trigger and FastAPI reject unattributed mutations. | PASS |
| TC-07 | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_02_worm_audit_log_immutability | 2026-10-17T21:20:40.096811+00:00 | 0.0042 | Verified UPDATE, DELETE, and TRUNCATE on part11_audit_log unconditionally raise 21 CFR Part 11 WORM Violation. | PASS |
| TC-08 | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_03_electronic_signature_rejects_invalid_credentials | 2026-10-17T21:20:40.112150+00:00 | 0.0911 | Verified POST /api/v1/event-frames/{id}/sign rejects invalid password with HTTP 401. | PASS |
| TC-09 | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_04_electronic_signature_approval_and_hmac | 2026-10-17T21:20:40.209120+00:00 | 0.2172 | Verified dual-challenge electronic signature execution, HMAC-SHA256 binding, and audit trail logging. | PASS |
| TC-10 | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_05_cryptographic_hash_chain_verification | 2026-10-17T21:20:40.434308+00:00 | 0.0758 | Verified 100% of part11_audit_log records form an unbroken SHA-256 hash chain from GENESIS_0000000000000000. | PASS |
| TC-11 | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_06_multi_protocol_telemetry_ingestion | 2026-10-17T21:20:40.518892+00:00 | 0.007 | Verified active telemetry ingestion across all 5 multi-protocol tags (101, 102, 103, 301, 401). | PASS |
| TC-12 | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_07_bad_quality_exclusion_in_rollup | 2026-10-17T21:20:40.533897+00:00 | 0.0373 | Proved mv_sensor_rollup_1m excludes quality=0 fault spike (9999.0), computing exact GMP rollup avg=37.5, max=38.0. | PASS |
| TC-13 | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_08_datalink_linear_and_locf_step_interpolation | 2026-10-17T21:20:40.586473+00:00 | 0.0575 | Verified exact linear midpoint (37.0) on BR201_TEMP_PV and integer LOCF ([1, 1, 2]) on BR201_AGIT_STATE. | PASS |
| TC-14 | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_09_lttb_downsampling_preserves_local_peak | 2026-10-17T21:20:40.659313+00:00 | 0.0211 | Verified ClickHouse LTTB 10x downsampling (100 -> 10 points) preserved exact peak magnitude (845.75 mAU) and timestamp (12:00:50Z). | PASS |
| TC-15 | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_10_event_frame_excursion_and_parquet_lakehouse_export | 2026-10-17T21:20:40.686786+00:00 | 0.2702 | Verified Event Frame evaluation and Apache Parquet export with 100% row/value fidelity via pyarrow. | PASS |
| TC-16 | gxp-validation-package/tests/test_04_ai_agent_and_mcp.py::test_oq_11_pydantic_ai_deviation_investigator_and_mcp | 2026-10-17T21:20:40.964214+00:00 | 0.4761 | Verified Pydantic AI gxp_deviation_investigator_agent produces a validated DeviationInvestigationBrief with part11-forensics capability and gxphistorian-mcp manifest. | PASS |
| TC-17 | gxp-validation-package/tests/test_04_ai_agent_and_mcp.py::test_oq_12_pydantic_ai_part11_human_in_the_loop_tag_approval | 2026-10-17T21:20:41.446523+00:00 | 0.2313 | Verified Pydantic AI ApprovalRequired (DeferredToolRequests / DeferredToolResults), args_validator ModelRetry recovery, and 21 CFR Part 11 WORM audit logging. | PASS |
2. Live 21 CFR Part 11 Cryptographic Audit Trail Evidence (part11_audit_log)
| Audit ID | Recorded At (UTC) | Actor ID | Change Reason | Target Table | Record PK | Op | prev_hash Prefix | row_hash Prefix |
|---|---|---|---|---|---|---|---|---|
1 | 2026-10-17 12:19:48.970102+00 | system_bootstrap | Initial GxP Qualification Seed | asset_templates | 1 | INSERT | GENESIS_000000000000... | 0980573d8a9a41b46f29... |
2 | 2026-10-17 12:19:48.971843+00 | system_bootstrap | Initial GxP Qualification Seed | asset_templates | 2 | INSERT | 0980573d8a9a41b46f29... | 0d36bf6886f6f8a95f53... |
3 | 2026-10-17 12:19:48.973299+00 | system_bootstrap | Initial GxP Qualification Seed | assets | 1 | INSERT | 0d36bf6886f6f8a95f53... | 1e50a1c48c7a31faf35e... |
4 | 2026-10-17 12:19:48.973792+00 | system_bootstrap | Initial GxP Qualification Seed | assets | 2 | INSERT | 1e50a1c48c7a31faf35e... | 4d8f96f4dc5b6270fb5d... |
5 | 2026-10-17 12:19:48.974952+00 | system_bootstrap | Initial GxP Qualification Seed | tags | 101 | INSERT | 4d8f96f4dc5b6270fb5d... | 01418e53ac7eea428007... |
6 | 2026-10-17 12:19:48.975413+00 | system_bootstrap | Initial GxP Qualification Seed | tags | 102 | INSERT | 01418e53ac7eea428007... | 31473f02919047f446ad... |
7 | 2026-10-17 12:19:48.975542+00 | system_bootstrap | Initial GxP Qualification Seed | tags | 103 | INSERT | 31473f02919047f446ad... | fa878c93ab51508a0788... |
8 | 2026-10-17 12:19:48.975672+00 | system_bootstrap | Initial GxP Qualification Seed | tags | 301 | INSERT | fa878c93ab51508a0788... | 8604eddc066bba1342f4... |
9 | 2026-10-17 12:19:48.975793+00 | system_bootstrap | Initial GxP Qualification Seed | tags | 401 | INSERT | 8604eddc066bba1342f4... | 4ef258b9cff9b1d15a01... |
10 | 2026-10-17 12:32:12.972505+00 | event_frame_engine | Automated ISA-88 Event Frame Limit Evaluation | event_frames | 8217b8ad-b909-4b03-8922-aab83751609f | INSERT | 4ef258b9cff9b1d15a01... | acf2a3d241e85bb1f95a... |
11 | 2026-10-17 12:32:12.982852+00 | event_frame_engine | Automated ISA-88 Event Frame Limit Evaluation | event_frames | d74b43a4-3dab-4181-8c51-8c906f92f451 | INSERT | acf2a3d241e85bb1f95a... | bc05d6786059961bfe1c... |
12 | 2026-10-17 12:32:18.018756+00 | event_frame_engine | Automated ISA-88 Event Frame Limit Evaluation | event_frames | 8217b8ad-b909-4b03-8922-aab83751609f | UPDATE | bc05d6786059961bfe1c... | 48bf097edae6380c088c... |
13 | 2026-10-17 12:32:18.05622+00 | event_frame_engine | Automated ISA-88 Event Frame Limit Evaluation | event_frames | ca9fcee4-ed1b-4326-9857-ecc8dfd3fabf | INSERT | 48bf097edae6380c088c... | 3888ca2b4406050e5b12... |
14 | 2026-10-17 12:32:23.09222+00 | event_frame_engine | Automated ISA-88 Event Frame Limit Evaluation | event_frames | 8217b8ad-b909-4b03-8922-aab83751609f | UPDATE | 3888ca2b4406050e5b12... | 3ac605d224d849b21779... |
15 | 2026-10-17 12:32:23.100325+00 | event_frame_engine | Automated ISA-88 Event Frame Limit Evaluation | event_frames | ca9fcee4-ed1b-4326-9857-ecc8dfd3fabf | UPDATE | 3ac605d224d849b21779... | 2b553f3b8800131fd39d... |
3. Live Signed ISA-88 Event Frames (event_frames)
| Event ID | Event Name | Template | Severity | Start Time | Signed By | Signature Meaning | HMAC-SHA256 Signature Hash |
|---|---|---|---|---|---|---|---|
2afff150-dbad-4919-b3df-3e5d1250d988 | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 21:05:22.193+00 | qa_approver_01 | Batch Release Approval | 9de08871f6bf6288d58ded3b... |
0fe19399-8c0c-4379-bb55-583871f7d448 | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 16:40:19.789+00 | qa_approver_01 | Batch Release Approval | 705d6b4d11ed782087a688a1... |
12006ce1-5502-461a-9cb4-3ec7efcc5e46 | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 13:20:17.952+00 | qa_approver_01 | Batch Release Approval | eb4025c9ecc02a2b7118cd50... |
f5aff8cb-2d04-4fc8-b854-ff8c63943e88 | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 12:26:14.459+00 | qa_approver_01 | Batch Release Approval | ce044436f26c50818761708b... |
d74b43a4-3dab-4181-8c51-8c906f92f451 | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 12:26:14.459+00 | qa_approver_01 | Excursion Investigation Reviewed | 0ab77f66b7de5503633a3213... |
8217b8ad-b909-4b03-8922-aab83751609f | USP_Batch_2026_BR201_001 | Fermentation_Batch | INFO | 2026-10-17 12:25:32.453+00 | UNSIGNED | - | -... |
ef356048-497b-40ed-8e88-ec5f1861e2ec | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 21:04:21.185+00 | UNSIGNED | - | -... |
6c6c18ea-685d-4e3f-a4f4-7f29b4a26357 | BR201_TEMP_PV_High_Limit_Excursion | Temp_Excursion | CRITICAL | 2026-10-17 21:02:22.167+00 | UNSIGNED | - | -... |
Document 07 — Bidirectional Requirements Traceability Matrix (RTM)
Document ID: RTM-GXPHISTORIAN-2026-007 | Generated (UTC): 2026-10-17T21:20:51Z | Coverage: 100.0% (17/17 URS Verified, 0 Gaps)
| URS ID | Requirement Title | FRS / Design Spec | GAMP Cat. | CSA Risk | Automated Pytest Node ID | Executed Timestamp (UTC) | Verdict |
|---|---|---|---|---|---|---|---|
| URS-001 | Multi-Service Container Infrastructure & Non-Default GxP Security Credentials | FRS-INFRA-001 | Category 3 / Category 4 | Medium | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_01_services_and_non_default_credentials | 2026-10-17T21:20:40.011342+00:00 | PASS |
| URS-002 | PostgreSQL 16 Control Plane Extensions & Core GxP Schema Qualification | FRS-PG-002 | Category 4 | Medium | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_02_postgres_extensions_and_schema | 2026-10-17T21:20:40.029800+00:00 | PASS |
| URS-003 | ClickHouse Lossless Time-Series Compression Codecs Verification | FRS-CH-003 | Category 4 | Low | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_03_clickhouse_compression_codecs | 2026-10-17T21:20:40.042621+00:00 | PASS |
| URS-004 | ISA-95 ltree Asset Hierarchy & ClickHouse In-Memory Dictionary Synchronization | FRS-DICT-004 | Category 4 | Medium | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_04_clickhouse_postgres_dictionary_sync | 2026-10-17T21:20:40.054417+00:00 | PASS |
| URS-005 | Stage 1 Edge Exception/Deadband Filtering & Disk WAL Store-and-Forward Buffer | FRS-EDGE-005 | Category 4 / Category 5 | High | gxp-validation-package/tests/test_01_iq_infrastructure.py::test_iq_05_edge_collector_wal_and_deadband | 2026-10-17T21:20:40.056478+00:00 | PASS |
| URS-006 | 21 CFR Part 11 Mandatory Attribution Context Enforcement | FRS-P11-006 | Category 5 | High | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_01_reject_mutations_without_actor_or_reason | 2026-10-17T21:20:40.084358+00:00 | PASS |
| URS-007 | 21 CFR Part 11 §11.10(e) WORM Audit Trail Immutability | FRS-P11-007 | Category 5 | High | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_02_worm_audit_log_immutability | 2026-10-17T21:20:40.100984+00:00 | PASS |
| URS-008 | 21 CFR Part 11 §11.200 Electronic Signature Rejection of Unauthorized Credentials | FRS-P11-008 | Category 5 | High | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_03_electronic_signature_rejects_invalid_credentials | 2026-10-17T21:20:40.203279+00:00 | PASS |
| URS-009 | 21 CFR Part 11 §11.50 & §11.70 Electronic Signature Manifestation & HMAC Binding | FRS-P11-009 | Category 5 | High | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_04_electronic_signature_approval_and_hmac | 2026-10-17T21:20:40.426290+00:00 | PASS |
| URS-010 | 21 CFR Part 11 Cryptographic SHA-256 Audit Hash-Chain Verification | FRS-P11-010 | Category 5 | High | gxp-validation-package/tests/test_02_oq_part11_compliance.py::test_oq_05_cryptographic_hash_chain_verification | 2026-10-17T21:20:40.510156+00:00 | PASS |
| URS-011 | Multi-Protocol Telemetry Ingestion (OPC UA, BACnet/IP, Siemens S7, Legacy OPC DA) | FRS-HIST-011 | Category 4 | Medium | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_06_multi_protocol_telemetry_ingestion | 2026-10-17T21:20:40.525899+00:00 | PASS |
| URS-012 | Bad OPC Signal Quality (quality < 192) Exclusion from GMP Rollups | FRS-HIST-012 | Category 5 | High | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_07_bad_quality_exclusion_in_rollup | 2026-10-17T21:20:40.571230+00:00 | PASS |
| URS-013 | PI DataLink Parity: Linear Interpolation vs. Discrete Step (LOCF) Sampling | FRS-DL-013 | Category 5 | High | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_08_datalink_linear_and_locf_step_interpolation | 2026-10-17T21:20:40.643979+00:00 | PASS |
| URS-014 | PI Vision PlotValues Parity: LTTB Visual Downsampling Peak Preservation | FRS-LTTB-014 | Category 4 / Category 5 | Medium | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_09_lttb_downsampling_preserves_local_peak | 2026-10-17T21:20:40.680383+00:00 | PASS |
| URS-015 | ISA-88 Event Frame Excursion Capture & Zero-ETL Apache Parquet Lakehouse Export | FRS-LAKE-015 | Category 5 | High | gxp-validation-package/tests/test_03_oq_historian_core.py::test_oq_10_event_frame_excursion_and_parquet_lakehouse_export | 2026-10-17T21:20:40.957006+00:00 | PASS |
| URS-016 | Pydantic AI Advisory Deviation Investigator & Read-Only Model Context Protocol (gxphistorian-mcp) | FRS-AI-016 | Category 5 | High | gxp-validation-package/tests/test_04_ai_agent_and_mcp.py::test_oq_11_pydantic_ai_deviation_investigator_and_mcp | 2026-10-17T21:20:41.440286+00:00 | PASS |
| URS-017 | Pydantic AI Human-in-the-Loop 21 CFR Part 11 Deferred Tool Approval Gate | FRS-AI-017 | Category 5 | High | gxp-validation-package/tests/test_04_ai_agent_and_mcp.py::test_oq_12_pydantic_ai_part11_human_in_the_loop_tag_approval | 2026-10-17T21:20:41.677813+00:00 | PASS |
Document 08 — Validation Summary Report (VSR) & Release Certificate
Document ID: VSR-GXPHISTORIAN-2026-008 | Release Date (UTC): 2026-10-17T21:20:51Z | System Release Status: QUALIFIED FOR GxP PRODUCTION USE
1. Executive Qualification Summary
The GxPHistorian GxP Industrial Historian (ClickHouse 24.8 Telemetry Archive, PostgreSQL 16 Part 11 Control Plane, Telegraf 1.32 Edge Collector, FastAPI Control Plane + Pydantic AI / gxphistorian-mcp, and Grafana 11.2 HMI) has completed all Installation Qualification (IQ), Operational Qualification (OQ), and Performance Qualification (PQ) activities in accordance with VMP-GXPHISTORIAN-2026-001.
| Metric | Target Acceptance Criterion | Actual Executed Result | Status |
|---|---|---|---|
| User Requirements Coverage (URS-001..017) | 100% Bidirectional Traceability | 17 / 17 Requirements Mapped & Verified | PASS |
Automated IQ/OQ Test Suite (pytest) | 100% Pass Rate (0 Failures) | 17 / 17 Passed (100.0%) | PASS |
| 21 CFR Part 11 Cryptographic Hash Chain | chain_valid == true from GENESIS | 7336 Audit Records Verified (head: eb0b5d30764ae40941f9a745...) | PASS |
| Validation Deviations / CAPAs | 0 Open Critical/Major Deviations | 0 Deviations Encountered | PASS |
2. Deviation & Non-Conformance Log
- Open Deviations:
0 - Closed Deviations:
0 - All 17 IQ/OQ test cases passed on execution without protocol exceptions or data integrity discrepancies.
3. Statement of Fitness for Intended GxP Use
Based on the objective cryptographic evidence captured in oq_execution_results.json and the unbroken SHA-256 WORM audit trail in part11_audit_log, the GxPHistorian GxP Industrial Historian is hereby declared fit for its intended GxP manufacturing use under FDA 21 CFR Part 11, EU GMP Annex 11, and ISPE GAMP 5 (Second Edition).
4. 21 CFR Part 11 §11.50 Electronic Release Signatures
| Signer Role | Printed Name | User ID | Signature Meaning | Signed Timestamp (UTC) | Cryptographic Verification |
|---|---|---|---|---|---|
| System Owner (IT/OT) | Sarah Jenkins | operator_01 | System Technical Qualification & Operational Handover Approval | 2026-10-17T21:20:51Z | Verified via pgcrypto & Part 11 Audit Chain |
| Quality Assurance (QP) | Dr. Marcus Vance | qa_approver_01 | Final QA Validation Release & 21 CFR Part 11 Compliance Approval | 2026-10-17T21:20:51Z | HMAC-SHA256 Bound (eb0b5d30764ae40941f9a7453d85d9c4...) |