Serving models to a mainframe core
A COBOL transaction program can call a model without hosting one. The model returns evidence — a score and a confidence — and the host keeps the decision. Change the model and the interface holds.
The separation principle
The model never adjudicates. COBOL owns eligibility, benefit rules, ledger state and the audit record; the inference runtime owns the graph. Below the confidence floor, or on any transport failure, the host rules decide — so a model outage never becomes a claims outage.
Export to ONNX first
A .onnx file is a serialized protocol buffer: graph topology and weight tensors, no executable code. It crosses a security boundary far more safely than a pickled Python object, which executes on load. Screen exportability before a model reaches architecture review.
optimum-cli export onnx \ --model mistralai/Mistral-7B-Instruct-v0.3 \ --task text-generation-with-past out/ osai verify out/model.onnx --parity 1e-4 --quantize int8
The C-ABI bridge
For the co-located pattern, the compiled graph and the transaction program share one address space. The copybook is generated into a packed C struct — alignment and numeric conversion are the parts that actually break, so write and test the bridge on a workstation with GnuCOBOL before it reaches an LPAR.
typedef struct { /* copybook aligned */
char member_id[10]; /* PIC X(10) */
char claim_id[12]; /* PIC X(12) */
double claim_amount; /* COMP-2 */
int32_t proc_code; /* COMP */
} CobolClaimRecord;
void SCORE_RECORD(CobolClaimRecord* r, float* score) {
*score = run_onnx_model(r);
}Keep the tokenizer in the graph
Never re-implement WordPiece or BPE in the host language — two implementations of one vocabulary drift apart. Export the tokenizer as a custom operator so the host passes a string and reads a score, and inference tokenization is identical to training by construction.
Log for the audit years later
Every prediction is stored with the input record state and the model version that produced it, in an immutable trail. Feature attributions are retained for scored records; language-model recommendations must cite the criteria they matched.