mlx-community/Mistral-7B-Instruct-v0.3-4bit

text-generationmlxapache-2.07.3B params · updated Jun 2024

Mainframe serving

Can a COBOL transaction program call this checkpoint, and under which transport? The model returns evidence; the host keeps the decision.

Export status

MLX

Quantized for the Apple Silicon runtime

Recommended pattern

Pattern 02

API gateway

Latency class

15–50 ms

ONNX artifact

Not measured

measured once the harness posts results

Screening note

Served by an inference service inside an enclave under an executed agreement. Identifiers are tokenized at the boundary.

Unstructured text in, recommendation out — served off-platform behind the de-identifying gateway

Tokenized egress over mTLS; executed agreement required

Repository targets the MLX runtime

osai-screening/1.0.0 · verdict 9ed68945768b3aa9…

Serving controlskill switch offshadow onconfidence floor 0.85GET /serving-status

Promotion gates

Gate	What it checks	State
Structural check	Graph loads; node inputs, outputs and shapes resolve	Not applicable
Numerical parity	Same inputs agree with the source framework within 1e-4	Not applicable
Eight-bit quantisation	INT8 weights with parity re-run afterwards	Not applicable
Shadow cycle	Full cycle scored against live traffic with no influence on outcomes	Not applicable

Interface contract

The copybook is generated, versioned and never hand-edited on either side. Entry point SCORE_MISTRAL_7B_INSTRUCT.

Generated copybookmodel record · contract

      * GENERATED BY osai-copybook-gen/1.0.0 — DO NOT EDIT (STANDARD S2)
      * COPYBOOK FOR mlx-community/Mistral-7B-Instruct-v0.3-4bit
      * REVISION a4b8f870474b  PATTERN 02  ENTRY SCORE_MISTRAL_7B_INSTRUCT
       01  SCORE-MISTRAL-7B-INSTRUCT-REQUEST.
           05  REQ-RECORD-ID          PIC X(12).
           05  REQ-TEXT-LEN           PIC S9(4) COMP-5.
           05  REQ-TEXT               PIC X(4000).
       01  SCORE-MISTRAL-7B-INSTRUCT-RESPONSE.
           05  RESP-SCORE             COMP-1.
           05  RESP-CONFIDENCE        COMP-1.
           05  RESP-LABEL             PIC X(16).
           05  RESP-MODEL-VERSION     PIC X(16).
           05  RESP-STATUS            PIC S9(4) COMP-5.

SCORE-MISTRAL-7B-INSTRUCT-REQUEST / OsaiRequest4014 bytes

COBOL	C	Kind	Offset	Bytes	Note
REQ-RECORD-ID	record_id	alnum(12)	0	12	caller correlation key, logged with the inference
REQ-TEXT-LEN	text_len	binary16	12	2	bytes used in REQ-TEXT
REQ-TEXT	text	alnum(4000)	14	4000	de-identified text; tokenized inside the graph

SCORE-MISTRAL-7B-INSTRUCT-RESPONSE / OsaiResponse42 bytes

COBOL	C	Kind	Offset	Bytes	Note
RESP-SCORE	score	float32	0	4	model output; evidence, not a decision
RESP-CONFIDENCE	confidence	float32	4	4	compared against the confidence floor by the host
RESP-LABEL	label	alnum(16)	8	16	e.g. REVIEW or APPROVED — never DENY
RESP-MODEL-VERSION	model_version	alnum(16)	24	16	revision that produced the score (standard S4)
RESP-STATUS	status	binary16	40	2	0 scored · 4 below floor · 8 transport or runtime failure — 4 and 8 fall through to host rules

CLAIMPROC.cob — the callergenerated by the screening service

       IDENTIFICATION DIVISION.
       PROGRAM-ID. OSAICALL.
       DATA DIVISION.
       WORKING-STORAGE SECTION.
           COPY OSAICPY.
       77  CONFIDENCE-FLOOR     COMP-1 VALUE 0.85.
       PROCEDURE DIVISION.
           EXEC CICS LINK PROGRAM('BAQEXEC')
             COMMAREA(SCORE-MISTRAL-7B-INSTRUCT-REQUEST)
             LENGTH(LENGTH OF SCORE-MISTRAL-7B-INSTRUCT-REQUEST)
             RESP(WS-CICS-RESP)
           END-EXEC
           IF WS-CICS-RESP NOT = DFHRESP(NORMAL)
             MOVE 8 TO RESP-STATUS
           END-IF
           EVALUATE TRUE
             WHEN RESP-STATUS NOT = 0
               PERFORM APPLY-EXISTING-HOST-RULES
             WHEN RESP-CONFIDENCE < CONFIDENCE-FLOOR
               PERFORM APPLY-EXISTING-HOST-RULES
             WHEN RESP-LABEL = 'REVIEW'
               PERFORM HOLD-FOR-REVIEW
             WHEN OTHER
               PERFORM CONTINUE-ADJUDICATION
           END-EVALUATE
           PERFORM WRITE-AUDIT-RECORD
           GOBACK.

The C-ABI wrapper is on the Usage tab under “C bridge”. Test the bridge on a workstation with GnuCOBOL before it reaches an LPAR — struct packing and numeric conversion are what actually break, and they are identical on both targets.

Standards this deployment inherits

Download ML-BOM

S1Models ship as ONNX artifacts. No ML framework runtime is installed on z/OS.

S2The copybook is the interface contract — generated, versioned, never hand-edited on either side.

S3Latency class selects the pattern. Team preference and vendor enthusiasm do not.

S4Every inference is logged with the record state and model version that produced it.

S5PHI leaves the platform only tokenized, over mTLS, to an endpoint under an executed BAA.

S6A model influences payment only after a full shadow cycle and with a tested fallback path.