Streaming data manager
Real-time feeds beside the batch pipeline — admissions and discharges, lab results, claims remittances. Streaming has no clean-run boundary, so each partition keeps its own checkpoint with explicit gap accounting, never sharing the bulk pipeline's watermark. A gap is surfaced loudly rather than interpolated.
Partitions
| Feed | Partition | Checkpoint | Latest offset | Lag | Events today | Last event | State |
|---|---|---|---|---|---|---|---|
| ADT admissions/discharges | 0 | 8,842,310 | 8,842,310 | 0 | 1,204 | 13:58 | healthy |
| ADT admissions/discharges | 1 | 8,790,522 | 8,790,522 | 0 | 1,187 | 13:58 | healthy |
| ADT admissions/discharges | 2 | 8,811,948 | 8,811,990 | 42 | 1,163 | 13:51 | gap — attention |
| Offsets 8811949-8811990 unacknowledged after broker restart. Surfaced loudly, never interpolated - an interpolated gap is a silent data-integrity failure that looks like a healthy pipeline. | |||||||
| Claims 835 remittance | 0 | 902,114 | 902,114 | 0 | 214 | 12:40 | healthy |
| ORU lab results | 0 | 4,410,221 | 4,410,221 | 0 | 3,388 | 13:59 | healthy |
| ORU lab results | 1 | 4,398,710 | 4,398,710 | 0 | 3,251 | 13:59 | healthy |
Source EMR lookup — pull one patient now
Between bulk runs, new patients exist at the source that PHI AI has not seen. Search the source EMR directly and import an individual patient over the per-patient read path — the same path bulk-less vendors force for entire populations.
An individual import runs read → encrypt → store → index for one patient and lands in your session's sandbox: the patient appears in Patients & charts, the import is audited (ingest.patient_imported), and the next reconciliation on the Bulk import manager counts them.