The CSM Fixed Macro — Architecture & Plan (read this before any 12d macro work)
**Status:** authoritative macro design, confirmed with Abdullah 2026-05-31. Complements
the product vision (`cm_csm_adac_aspec_product_vision.md` §0B, §1A, §1B, §1C). If this
doc and an older per-client-macro assumption conflict, THIS doc wins.
> **👉 Want the picture, not the prose?** Open `docs/CSM_Macro_Vision_Explainer.html`
> in a browser. It is the interactive, plain-language version of THIS document —
> the panel mockup with real UnityWater data, a click-through worked example, and
> the source-kind legend. Use it to onboard, then come back here for the build detail.
§00. The macro at a glance (read this first — resolves three recurring confusions)
Three things were described inconsistently across docs and chats. They are now locked
here. If any older note disagrees, THIS section wins.
**1. ONE fixed macro, two files — not three.**
`CSM_Apply_Profile` is a single fixed macro that works for every client and every job.
It has exactly **two file boxes** at the top:
| File box | What it is | Macro access |
|---|---|---|
| ① **Client Settings File** | CSM-authored, per client/schema: *which fields you must input* + *how the macro computes everything else*. Maps to `project_runtime_profile.12d_runtime.txt` + `csm_field_input_spec.csv` + `csm_runtime_actions.csv`. | read-only |
| ② **Project Sub-Profile** | Your per-job answers + the names of the 12d objects you picked. Maps to `CSM_client_subprofile.txt`. | read **and** write |
There is **no third "seeded code file"** box (the original UI shell prototype had one).
In production the **seeded codes come from scanning the selected source model**, not a file.
**2. The two "modes" are TABS of one panel — not two macros.**
The panel has these tabs:
| Tab | What it does | Maps to vision concept |
|---|---|---|
| **Seeded Setup** | Scan the survey → list the client features actually present → review each group's attributes → bulk-apply group-safe values. | the Group Manager / review front-end (§4A) |
| **Element Review** | Per-element overrides for values that must stay unique. | per-element exceptions (§4A buckets) |
| **Project Values** | Fill the office/project values and pick the 12d references the macro needs. This tab IS the field-input spec made visible. | "which fields the user must input" (§3) |
| **Run Evidence** | What the last run computed + audit proof. | evidence/log (§4 step 7) |
| **Settings** | Apply mode, GDI cap, output prefix. | — |
So "group review/bulk apply" and "per-project value entry" are **not** competing designs;
they are the **Seeded Setup** and **Project Values** tabs of the same panel.
**3. Flat-grid vs tree is a navigation choice over the SAME data — not a redesign.**
The group list can render as a flat `GridCtrl_Box` (the approved mockup,
`CSM_Apply_Profile_UI_Mockup_Improved.png`) or as a `Tree_Box` of group→element nodes
(the 2026-06-02 tree-first build, see `TWO_FILE_CONVERTER_SETTINGS_ARCHITECTURE.md`).
Both read the same scanned-and-grouped data; the navigation widget is the only difference.
**The tree/grid is navigation; the right-pane grids are the editing surface.** Do not treat
the tree-first doc and the mockup as conflicting visions — they are two views of one panel.
0. Interactive UI is TREE-FIRST (IMPLEMENTED 2026-06-02) — read before §2
The operator-facing review panel in `exports/CSM_Apply_Profile.4dm` is **tree-first**,
not a flat grid and not resolver-first. This is the current, compiled design
(`CSM_Apply_Profile.4do`, clean `.4dl`, 0 unknown calls, 2026-06-02):
- **Left `Tree_Box`** (`Create_tree_box`, id 2571) of **group nodes** — one per
seeded element/group name (`csm_seeded_group_name`) — each with **element child
nodes** (`Create_tree_page`, id 2577).
- **Right pane is native per page**: each `Tree_Page` carries its own appended
widgets (12d swaps them on node click). A **group node** shows element count /
missing / blocked / changed counts, an element-list `GridCtrl_Box`, and a
**group-safe bulk apply** (one named attribute → all elements in that group,
via `csm_vset`). An **element node** shows that element's attribute grid with
**Attribute | Current value | Final value | Source/bucket** columns (current =
value on the element now; final/source resolved from the CSM settings sidecar
where available, else blank + storage bucket) and a **per-element override**
(`csm_vset` to that element only).
- **Apply buttons live on each page** with an index-encoded command
(`applygrp_<i>` / `applyel_<k>`); the event loop dispatches them — no unverified
tree-selection-query call is used.
- **Scan rebuilds the tree** (teardown + rebuild, position preserved), matching
the verified reference `Element_Attribute_Tree_Panel_v2.4dm`.
- **GDI cap:** the macro manual warns many `Tree_Page`s crash 12d, so element
child-nodes are capped (`CSM_MAX_ELEM_NODES`); every element still appears as a
row in its group page's element grid.
- The fixed **runtime/apply/evidence engine** (the `--csm-autorun` path in
`main()`, `csm_vget`/`csm_vset` super-vertex writes) is **unchanged**. The
pre-2026-06-02 flat tabbed panel is retained unreferenced as
`csm_show_flat_review_panel_legacy` for rollback only.
**Removed assumptions:** resolver-first ordering, client-/Veris-code-first
grouping, and flat-grid-as-primary editing. Grouping is by seeded element/group
name; the tree is navigation; the right pane is the editing surface. See
`docs/TWO_FILE_CONVERTER_SETTINGS_ARCHITECTURE.md` and
`_planning/CSM_UI_UX_WORKFLOW_REBUILD_PLAN.md` for the full UI contract.
**Verification status (2026-06-02):**
- ✅ Clean compile (`CSM_Apply_Profile.4do` fresh, `.4dl` zero bytes); 0 unknown calls.
- ✅ Page-nested action buttons route through `Wait_on_widgets` — verified against the
working reference `WIP_macros/Set_Global_Attributes_Panel/Set_Global_Attributes_Panel_tree_rebuild.4dm`
(`add_choice_leaf` appends a `Button` with a dynamic command onto a `Tree_Page`; its
event loop reads that exact command via `Wait_on_widgets` + `is_known_choice(cmd)`).
So `applygrp_<i>` / `applyel_<k>` dispatch is a proven pattern, not an inference.
- ✅ Safety contract honoured by construction: group bulk-apply writes ONLY the one named
group-safe attribute to the group's elements (all other/unique attributes untouched);
per-element override writes one attribute to one element. Nothing is overwritten without
an explicit typed attribute+value and an explicit Apply click.
- ⏳ **Live 12d UI/selection screenshot + GDI-at-scale: BLOCKED** at build time — 12dAITester
transport `blocked`/`degraded`/`stale`. Rerun the live UI proof when the tester is healthy.
**Scope of this iteration (honest, to avoid fake-green):** the element right pane shows
**Attribute | Current | Final | Source/bucket**. Current is the live element value;
Final/Source are resolved from the CSM settings sidecar (project defaults) where
available, else Final is blank and Source is the storage bucket (string/vertex) — this
matches goal item 5 ("...if available"). Deeper Final resolution (full
`csm_runtime_actions.csv` rule/value-map/formula evaluation per attribute) remains the
fixed runtime/apply engine's job and its evidence CSV; surfacing that live in the nav
pane is a later increment. The only gate that is environment-blocked (not a code gap)
is the **live 12d UI/selection screenshot + GDI-at-scale run**, pending tester health.
---
1. The one-paragraph picture
There is **ONE fixed 12d macro** (`CSM_Apply_Profile`). It works for **any client and
any project**. It is **the UI**: when it runs it builds its own panel — input boxes,
model boxes, TIN boxes, choice boxes — **dynamically**, from a CSM-authored settings
file, because every client/code needs different fields. The user fills in the per-project
values and picks the 12d references; the macro **saves those into a client sub-profile
file** (so re-running the project later just reloads them), then **does all the
calculations itself in 12dPL** and writes the attributes onto the survey elements.
**CSM does NOT generate 12dPL.** CSM generates **converter settings** (data/config). The
macro is fixed and stable; only the settings + sub-profile change per client/project.
`.12dattmf` / `.12dMetaConnex` are **last-resort adapters**, never the first move.
MetaConnex and Attribute Manipulator are still valuable **native 12d UI references**:
they show the complexity level 12d users already understand (tree navigation, selected
feature/attribute details, rules/actions, read/write files, active flags). CSM is building
its own client-schema-driven tool, not cloning or depending on MetaConnex as the engine.
---
2. The two files the macro reads (the "two file boxes")
┌─────────────────────────────┐
CSM ───▶│ CSM SETTINGS FILE (read-only) file box 2
│ the "converter settings" │
│ (a) WHICH fields the user must input │ authored by CSM,
│ + which widget each needs │ never edited in the macro
│ (b) HOW the macro calcs everything else │
└──────────────┬──────────────┘
│ macro reads it to BUILD THE PANEL + run the maths
▼
┌─────────────────────────────┐
│ CSM_Apply_Profile.4do │ ONE fixed macro = the UI + engine
│ builds dynamic panel ──▶ user fills/selects ──▶ runs calcs ──▶ writes attrs
└──────────────┬──────────────┘
│ macro reads AND writes it
▼
┌─────────────────────────────┐
user ──▶│ CLIENT SUB-PROFILE (read/write) file box 1
│ the user's per-project values: │ saved by the macro
│ entered values + selected model/TIN/poly │ reload on re-run = no re-entry
└─────────────────────────────┘File box 1 — CLIENT SUB-PROFILE (per project; macro reads AND writes)
- Holds the user's per-project answers: office-entered values, project default values,
and the **names** of the 12d references the user selected (models, TINs, strings,
polygons, alignments).
- The macro **writes** it when the user clicks save/run, and **reloads** it next time —
so re-running the same project is trivial (no re-entry).
- Maps to the existing `project_runtime_profile.csm.json` / `.locked.json` /
`project_runtime_profile.12d_runtime.txt` sidecar. The macro reads/writes the simple
line-pair sidecar (12dPL is safer with line-pairs/CSV than nested JSON).
- Store reference selections as **names** (Dynamic_Text), not handles — re-fetch the 12d
handle at run time via `Get_model(name)` etc. (Core: `Dynamic_Model` is not a 12dPL type.)
File box 2 — CSM SETTINGS FILE (per client/schema; read-only converter settings)
Two logical parts:
- **(a) Field-input spec** — the list of fields the user must input, each with: attribute
name, label, widget type (input/model/TIN/choice/string/polygon), choice list (if any),
required flag, default. **The macro builds its panel from this.** (This spec is the part
most likely MISSING today — see Gap, §6.)
- **(b) Calc / converter settings** — per attribute: its `source_kind` + the rule/formula/
value-map/reference definition the macro executes. Maps to the existing
`csm_runtime_actions.csv`, `csm_value_maps.csv`, `csm_rule_manifest.csv`.
---
3. THE KEY RULE — which fields land on the panel vs which the macro computes
This falls directly out of each attribute's `source_kind` (vision §0B + §1C). The dynamic
panel exposes an input ONLY for the source_kinds the user must supply; the macro computes
the rest from the settings file.
| `source_kind` | Who/what populates it | On the macro panel? | Widget |
|---|---|---|---|
| `capture_new` | **Surveyor in the field** (already in the `.12da`) | NO — macro reads it | — |
| `office_ingestion` | **Office user, in the macro** (per project) | **YES** | Input box (text/typed) |
| `project_default` | **User, in the macro** (per-project value) | **YES** | Input box / Choice box |
| `reference` | **User selects a 12d object** (model/TIN/string/polygon/alignment) | **YES** | Model box / TIN box / Select box |
| `spatial_lookup` | User selects the reference geometry; macro derives | **YES** (the reference) | Model/Polygon box |
| `derived_formula` | **Macro computes** (Z, bearing, surface-minus-Z, concat…) | NO | — |
| `conditional_rule` | **Macro computes** (IF/THEN over other attrs) | NO | — |
| `value_map` | **Macro translates** (Veris choice → client choice) | NO | — |
| `veris_attr` (legacy) | **Macro copies** from a Veris attr | NO | — |
| `ignore` | not needed | NO | — |
| `blocked_manual` | no executable path yet | NO — decision queue; agent final-review when evidence is enough | — |
So the panel = one control per `office_ingestion` / `project_default` / `reference` /
`spatial_lookup` field declared in the client's settings, grouped sensibly (e.g. a tab or
section per feature group). Everything else the macro derives. **That is exactly "which
fields the user needs to input and how the macro does the calcs."**
---
4. End-to-end flow
- **CSM (app)** exports for the client/project: the **CSM settings file** (field-input
spec + calc settings) and an initial **client sub-profile** (blank/defaults).
- **Operator runs `CSM_Apply_Profile`** in 12d. File box 2 → point at the CSM settings
file. File box 1 → point at (or create) the client sub-profile.
- **Macro builds the dynamic panel** from the field-input spec: the right input/model/TIN/
choice boxes for THIS client, pre-filled from the sub-profile if it already has values.
- **User fills values + selects references** (Design TIN, sewer-main model, package
polygon, asset owner, job number, …) → clicks Save/Run.
- **Macro saves** the entered values + selected reference names into the **client
sub-profile** (file box 1).
- **Macro runs all calcs in 12dPL** per the settings file: copies captured attrs, applies
value maps, runs conditionals, computes derived formulas, resolves references/spatial
lookups, applies project/office defaults — writes each attribute onto the elements.
- **Macro writes evidence** (which profile, which references, computed values, paths).
- **Re-run later:** reload the same sub-profile → panel pre-filled → run. No re-entry.
---
4A. Group-first seed and bulk-attribute workflow
Abdullah's grouping idea is part of the macro architecture. It is not just validation and
it is not a replacement for the fixed runtime engine.
Design contract:
Group Manager = review/grouping/bulk-entry front-end
CSM_Apply_Profile = fixed runtime apply/evidence engineMetaConnex is the layout/complexity reference, not the product boundary. The CSM grouping
front-end should learn from its left-tree/right-detail pattern, but it must remain
CSM-native and data-driven from CSM exports.
The Group Manager should help the operator answer one question:
Which selected 12d elements need the same editable client attributes, so one set
of values can be reviewed and applied to that group without corrupting assets
that only look similar?The fixed runtime macro still owns formulas, value maps, fallback chains, project
defaults, reference geometry, ATTMF metadata, and evidence.
The intended workflow is:
select source data
select client
load/seed that client's CSM schema package
resolve each element to the correct client feature
build groups from the required writable attribute signature
bulk apply values to each group
run the fixed CSM runtime/apply/evidence path
log exceptionsThe key rule is:
resolve first
group second
bulk apply third
validate/report lastDo not group directly by raw Veris code, raw client code, client feature name only,
attribute count only, or choice list only. Those keys are too weak and will mix assets
that look similar but need different client attributes.
Strict grouping key
The resolver must first work out what each selected 12d element actually is for the
selected client. The grouping key should include:
client
schema version
client feature code
Veris/field code
geometry type
match attribute
match value
required attribute paths
attribute types
choice-set IDs
source kind
runtime/reference requirements
field/office/manual/derived classificationUse full attribute paths where available, not just short names. 12d attributes can be
hierarchical and case-sensitive, and repeated names can appear in different contexts.
Two levels of grouping
Use two levels:
Level 1: asset resolution group
Level 2: bulk-entry form groupExample:
Level 1:
UnityWater / Sewer.Connection / SWSUG / Sub Type = House Connection
Level 2:
Manual fields required:
SurfaceLevel_m
InvertLevel_m
Use
Diameter_mm
Class
SO_Nearest_m
SO_Other_m
Sediment_TrapThis prevents different asset types from being mixed, while still allowing two asset
types with the same manual required fields to use the same bulk-entry form.
UnityWater proves why code-only grouping is unsafe
UnityWater uses the same Veris code for different client assets based on discriminator
attributes. Examples:
| Veris code | Match attribute | Match value | Client feature |
|---|---|---|---|
| `SWSUG` | `Sub Type` | `House Connection` | `Sewer.Connection` |
| `SWSUG` | `Sub Type` | `Gravity sewer` | `Sewer.NonPressurePipe` |
| `SWSUG` | `Sub Type` | `Rising Main` | `Sewer.PressurePipe` |
| `SSBOU` | `Sub Type` | `Lot Parcel` | `LotParcel` |
| `SSBOU` | `Sub Type` | `Road Reserve` | `RoadReserve` |
| `SSBOU` | `Sub Type` | `Water Course Reserve` | `WaterCourseReserve` |
Grouping only by `SWSUG` or `SSBOU` would be wrong. The macro must resolve the client
feature first, then group by required attributes.
UnityWater also has runtime reference branches, including cadastral and sewer-main
reference logic. Proving one branch does not prove the other. The group signature must
carry reference/runtime requirements so a bulk group does not hide a missing reference
binding.
Vida proves why grouping is valuable
Vida has a large runtime package: inspection attributes, runtime actions, value maps, and
value-map entries. Grouping by required attribute signature collapses repeated work. A
large set of Vida features can share the same required capture attributes, and several
underground service features can share another required-attribute signature.
The user should not edit hundreds of individual strings when one resolved group can be
reviewed and filled once, with exceptions logged separately.
Attribute buckets
Do not push every `capture_new` row into the surveyor or bulk prompt. The correct field
prompt gate is `field_capture_required` / `must_capture_in_field`, not `source_kind =
capture_new` by itself. A `capture_new` row can still represent office entry, runtime
input, formula, fallback, value-map, or macro-derived logic.
Every attribute in a resolved group should be classified into one of these buckets:
| Bucket | Meaning |
|---|---|
| Bulk manual entry | User can enter one value and apply it to all elements in the group. |
| Per-element manual entry | Required, but likely different per element, such as serial number or lot/asset number. |
| Field captured | Expected to already exist from field/controller data. |
| Office entered | Supplied by office user/project context. |
| Project default | Supplied once through the client sub-profile/runtime profile. |
| Fixed default | Comes from exported CSM settings. |
| Value mapped | Macro translates a source value to a client value. |
| Geometry derived | Macro derives from element geometry. |
| Reference derived | Macro derives from selected model/TIN/string/reference geometry. |
| Formula/conditional | Macro computes from formula/rule settings. |
| Blocked / unknown | Client needs it but no safe executable source is mapped yet. |
Only the bulk manual bucket should be applied blindly to every element in a group.
Practical panel design
The grouping panel should use this flow:
Source Box: selected raw survey data
Client Choice Box: Vida, UnityWater, MRBC, etc.
Schema/version/package selector
Scan button
Resolver:
read element name/code
read discriminator attributes
resolve to client feature
load required attributes
Group builder:
create group_id from resolved feature + attribute signature
Group grid:
tick
group_id
client feature
Veris code
match rule
element count
required count
missing count
blocked count
status
Attribute grid for selected group:
attribute path
type
required
choices
source kind
bucket/classification
bulk value
overwrite
notes
Buttons:
Preview group
Select group elements
Apply selected group
Apply all valid groups
Export reportUse `GridCtrl_Box` for the group and attribute tables. It is the verified dynamic rows and
fixed-column widget mechanism for 12dPL. Populate or resize grid data after `Show_widget`
where required by the GridCtrl pattern.
Reports and traceability
Each scan/apply run should produce:
CSM_Group_Report.csv
CSM_Group_Apply_Log.csv
CSM_Missing_Required_Attributes.csv
CSM_Blocked_Attributes.csvOptional trace attributes on each 12d element:
CSM/client
CSM/client_feature_code
CSM/group_id
CSM/group_status
CSM/last_appliedThese tags make reruns auditable and allow already-processed elements to be filtered or
reviewed.
MVP sequence
Build in phases, with a hard acceptance check at each phase:
| Phase | Scope | Acceptance check |
|---|---|---|
| 1. Scan/group | Select source model, select client, resolve each element to client feature, group by required attribute signature, export reports. | Group report is correct for Vida, UnityWater, MRBC, and one ADAC-style client. |
| 2. Bulk apply | Fill values for one selected group, write attributes, log changes. | Only bulk-manual attributes are written; exceptions are explicit. |
| 3. Runtime integration | Feed group values into the `CSM_Apply_Profile` runtime/evidence path. | Evidence distinguishes manual, formula, value-map, fallback, geometry/reference, skipped, blocked, and unresolved rows. |
Production integration must keep this split clear:
Group Manager = review/grouping/bulk-entry front-end
CSM_Apply_Profile = fixed runtime apply/evidence engineThe grouping front-end can generate bulk values or override data, but the final production
apply path should still feed the fixed runtime package so formulas, value maps, fallback
chains, project defaults, reference geometry, ATTMF metadata, and evidence are preserved.
Open questions for Abdullah
- Should the first real grouping proof be `UnityWater`, `Vida`, or both?
- Should two different client features with the same manual field list share one
bulk-entry form, or should every client feature always remain a separate editable group?
- Should the grouping tool write trace attributes such as `CSM/group_id` during scan, or
only after apply?
- Should group bulk values be passed into `CSM_Apply_Profile` through element attributes,
a generated override CSV, or the client sub-profile?
---
5. Worked example (one attribute, end-to-end) — Vida side-entry-pit depth
`VT_UTL_DEPTH` on Vida feature "308 Side entry pit", `source_kind=reference` +
`derived_fn=surface_minus_z`, `source_ref=TIN_A`:
- CSM settings file row says: VT_UTL_DEPTH comes from a TIN reference, formula = surface − Z.
- Field-input spec therefore declares ONE input: a **TIN box** labelled "Design TIN (TIN_A)".
- Panel shows that TIN box; user selects `Design Surface 2026-05-24`; macro saves
`reference.TIN_A=Design Surface 2026-05-24` to the sub-profile.
- At run: macro reads the captured pit at (502000, 6959000, z=10.0), looks up the TIN
surface at that XY (=11.25), computes 11.25 − 10.0 = **1.25**, writes `VT_UTL_DEPTH=1.25`.
- The surveyor never captured depth; the user never typed 1.25 — the macro derived it from
the user's one TIN selection + the settings file. Re-run a different project = same macro,
different sub-profile (different TIN), zero code change.
---
6. Current state, gap, and build plan
> **⚠️ Status caveat (2026-06-15 gap audit — `.planning/CSM_GAP_REPORT_2026-06-15.md`):**
> The "BUILT 2026-05-31" note below is correct for **Vida** (1 field: SURVEY_METHOD).
> But onboarding the new **AquaPath** client produced a **header-only**
> `csm_field_input_spec.csv` (0 data rows), so the macro had no fields to build its
> panel from. This is a per-client **exporter bug, not a design change** — the design
> (spec drives the panel) still stands. Two open blockers from that audit:
> **GAP-01** — `generate_field_input_spec()` emits no rows for clients whose inputs
> are `capture_new`/`office_ingestion`/`project_default`/`reference` (debug the filter);
> **GAP-02** — `macro_pack.py` still generates a per-client `.4dm` with hardcoded
> absolute paths, which violates "one fixed macro, CSM generates data not code" — retire it.
> **BUILT 2026-05-31 (Phase A + B — compiled & tested; live e2e pending tester):**
> - **Phase A DONE.** `generate_field_input_spec()` added to
> `csm/exporters/project_runtime_profile.py` (derives the spec from each
> attribute's `source_kind` per §3, deduped by input key). `runtime_package.py`
> now emits `csm_field_input_spec.csv`/`.jsonl` and registers it in the manifest
> (`counts.field_inputs`). Vida yields exactly **1 field: SURVEY_METHOD** (text) —
> its 80 "reference" mappings are `own_geometry`, needing no picker. Locked by
> `tests/test_field_input_spec.py` (5) + `tests/test_macro_subprofile_contract.py` (3).
> - **Phase B DONE (compiled 2026-05-31 23:04).** `CSM_Apply_Profile.4dm` now:
> uses real `File_Box` widgets for the sidecar/settings path and the client
> sub-profile path, reads the field-input spec, builds a **dynamic GridCtrl_Box**
> panel (one row per field, typed columns for text/model/source/TIN/choice), reads
> AND writes a **client sub-profile** (`CSM_client_subprofile.txt`, key=value —
> File Box 1) on **Save Values** or **Run**, and its `project_default` and runtime
> reference branches prefer the operator's saved per-project values over baked
> sidecar defaults. The old UnityWater-specific standalone pickers and text override
> fields were removed. Autorun accepts a 9th arg `<subprofile>` for headless runs.
> Panel prefill, Load Fields, Save Values, validation failures, and final Run all
> clear the 12d Output Window first and then write `CSM_Apply_Profile_output_window.xml`
> beside the sidecar package, so output-window parse errors are run-specific and can be
> inspected after launcher runs.
> - **Vertex attribute path DONE; UnityWater representative coverage closed.** Macro output
> writes now use `csm_vset` to write the super-vertex `Attributes` bag with
> `Get_super_vertex_attributes` / `Set_super_vertex_attributes`; source reads use
> `csm_vget` (vertex-first, element fallback only for old/source sample attrs). This is
> the required path for client output attributes. Model existence and element-level attrs
> are only transport/debug smoke. As of 2026-05-31 23:33, the UnityWater representative
> sample independent inspector reports `checked_values=58`, `issue_count=0`,
> `missing_required_count=0`, `stub_value_count=0`, `debug_attr_count=0`, and
> `type_mismatch_count=0`.
> - **Phase C PENDING (environment, not code).** The live e2e run through
> `CSM_Apply_Profile` is wired and ready but blocked: the 12dAITester transport is
> `blocked`/`stale` (helper visual channels down; `live_worker_transport` stale).
> Ready command once the tester is healthy:
> `CSM_Apply_Profile.4do --csm-autorun <sidecar> <surveyModel> "CSM " "" "" "" "" <subprofile>`
> then inspect `VT_UTL_DATA_ACQUISITION_METHOD` == the sub-profile SURVEY_METHOD.
**Exists today:** `exports/CSM_Apply_Profile.4dm/.4do` (the fixed macro — reads the
`.12d_runtime.txt` sidecar + `csm_runtime_actions.csv`, has a panel, applies rules, writes
evidence; MRBC smoke set an attribute). The Vida instruction package exists
(`csm_runtime_actions.csv`, `csm_value_maps.csv`, `csm_rule_manifest.csv`).
**Closed gaps as of 2026-05-31 23:04:**
- `csm_field_input_spec.csv` exists as exported data and drives the fixed macro panel.
- The dynamic panel uses `GridCtrl_Box` row data with typed columns rather than hardcoded
per-client controls.
- The macro uses real `File_Box` widgets for the sidecar/settings package and client
sub-profile, with explicit Save Values and Run save-back.
- Runtime reference values such as `TIN_A`, `CADASTRAL_A`, `ROAD_CL_A`, and
`SEWER_MAIN_A` are read from the dynamic grid/sub-profile by `input_key`, not from
UnityWater-specific standalone controls.
- Output attribute writes are vertex-native. The fixed macro, UnityWater shim, and
production exporter now write through `Get_super_vertex_attributes` /
`Set_super_vertex_attributes` instead of element-level `Set_attribute(elt, ...)`.
- Output-window evidence is current-run evidence. Macros call `Clear_console()` before
`Save_output_window_to_XML(...)`, so stale launcher errors do not poison fresh runs.
**Remaining design work:**
- Move from the current CSV sidecar package (`project_runtime_profile.12d_runtime.txt`
+ `csm_field_input_spec.csv` + calc CSVs) to a single optional
`csm_settings_<client>_<schema>.json` wrapper only if 12dPL parsing remains simple and
proven. Do not block the fixed macro on JSON.
- Add richer grouping/tabs from `panel_tab` when the grid becomes too dense.
- For new clients or real-project variants, keep client-specific action coverage honest:
every required output attribute must be populated in the **vertex Attributes bag** and
proved by the independent inspector. UnityWater's representative sample is now green;
do not generalize that to a new client/project without rerunning the proof.
**Build order (Phase 2 — gated on Phase 1 app perfection per vision §1493):**
a. CSM emits the field-input spec (derive from each attribute's `source_kind` per §3).
b. Macro builds the dynamic panel from it (approach per DYNAMIC_MACRO_PANEL_DESIGN.md).
c. Macro saves/reloads the client sub-profile.
d. Prove Vida end-to-end through `CSM_Apply_Profile` (not the per-client worker) on the
full-coverage synthetic survey → compliant deliverable.
**Do NOT** build per-client `.4dm` macros as the target (vision §1B: those are throwaway
prototype/compat shims). One fixed macro, data-driven.
---
7. One-line test for any agent
If you can trace one attribute from its `source_kind` → whether it appears on the panel →
which widget → how the macro computes/saves it (§3 + §5), you understand the macro. If you
propose generating a per-client macro or reaching for `.12dattmf` first, you do not.
---
8. Dynamic panel mechanism — recommended approach
**Verified base (what `CSM_Apply_Profile.4dm` already does today):** it builds a real panel
with `Create_panel(title,1)` → `Create_colour_message_box("Status")` → `Create_tab_box()`
(no args, per Core) → several `Create_input_box(label,status)` + one
`Create_source_box(label,status,Source_Box_Standard)` (the survey model) +
`Create_button("Run","run")`, grouped with `Create_vertical_group`/`Create_horizontal_group`
+ `Append(...)`, shown via `Wait_on_widgets`, read with `Get_data(widget)`, pre-filled with
`Set_data(widget,value)`. **The problem:** those input boxes are HARDCODED (sidecar, TIN
override, cadastral, sewer-main, output prefix) and references are typed as text, not picked.
**Recommended approach: NATIVE widgets created in a LOOP over the field-input spec** (not
`Run_slf_data` runtime XML — that fits driving built-in panels, not a bespoke per-client
input panel; native widgets read back cleanly via `Get_data` and the macro already uses
them). The macro:
- Reads File Box 2 (CSM settings) → the `field_input_spec` array.
- Groups fields by `panel_tab`; for each field, in a loop, creates the widget for its
`input_type` and `Append`s it to the tab/group, storing the handle in an array:
| field `input_type` (from §3 source_kind) | widget call |
|---|---|
| text / value (`office_ingestion`, `project_default`) | `Create_input_box(label, status)` |
| choice (`project_default` with choices) | `Create_choice_box(...)` — VERIFY signature live |
| model reference (`reference`/`spatial_lookup`) | `Create_source_box`/`Create_model_box` — see note |
| TIN reference (`reference`) | `Create_tin_box` — VERIFY exists live |
| survey source model | `Create_source_box(label, status, Source_Box_Standard)` (proven) |
- Pre-fills each widget from the client sub-profile (File Box 1) via `Set_data` if a saved
value exists (this is the easy-re-run path).
- `Wait_on_widgets`; on **Run/Save**, loops the handle array calling `Get_data(handle)` →
value; for model/TIN, `Get_data` returns the selected **name** (Text) → store the NAME
(`Dynamic_Text`, NOT `Dynamic_Model` per Core CORE-12D-0005; re-fetch via
`Get_model(name)` at apply time).
- Writes every value back to the client sub-profile (`File_open`+`File_write_line`,
`key=value` line-pairs), then runs the calcs (§3/§4) and evidence.
**VERIFIED 2026-05-31 (compiled via veris_runtime CLI against live 12d cc4d) — the
above "native widgets in a loop" idea was WRONG; here is what actually works:**
- ❌ **`Dynamic_Input_Box` / `Dynamic_Choice_Box` / `Dynamic_Widget` do NOT exist.** You
cannot store a variable-length array of widget handles. (Compile: "type not defined".)
- ✅ **The mechanism is `GridCtrl_Box`** — a table whose ROW COUNT is dynamic (data-driven),
with per-COLUMN widget types. This is how the dynamic panel is built. Verified call set
(see `_12D_MACRO_DATABASE/WIP_macros/GridCtrl_Demo/GridCtrl_Demo.4dm`, Lee 2026-05-28):
- `Widget cast(Widget w){return w;}` — REQUIRED to load typed widgets into `Widget cols[]`.
- `GridCtrl_Box Create_gridctrl_box(name, num_rows, num_cols, Widget cols[], show_nav, msg, w, h)` (ID 2393, 8-arg).
- `Insert_row(grid)` (2396) · `Delete_all_rows(grid)` (2409) · `Get_row_count(grid)` (2398).
- `Set_cell(grid, row, col, text)` (2400) · `Get_cell(grid, row, col, text&)` (2401) — **not** Set_data/Get_data.
- `Set_column_width(grid, col, px)` (2402) — call **after** `Show_widget`.
- Column widget per type: `Create_input_box(t,msg)` (text) · `Create_choice_box(t,msg)` (890; seed options via `Set_data(cb,n,opts[])`) · `Create_model_box(t,msg,GET_MODEL)` (848) · `Create_tin_box(t,msg,CHECK_TIN_EXISTS)` (962) · `Create_colour_box(t,msg)` (894).
- **As built (Vida):** one grid, cols `[Field(ro), Value(text), Hint(ro)]`, one row per spec
field. Choice/model/TIN columns are the migration path for reference-bearing clients
(e.g. UnityWater); Vida declares none, so they are compile-verified but not yet live-run.
- Re-confirm any new call via `query_12d_reference_index.py <Name>` (offline-safe) or
`progm_lookup` when the MCP is up.
Verified output/attribute rules
- **Output Window evidence:** call `Clear_console()` at the start of the macro branch, then
call `Save_output_window_to_XML(path)` before exit/error. Without the clear, the XML can
contain stale launcher errors from earlier runs and produce false blockers.
- **Client output attributes are vertex attributes.** Write required client output values to
the super-vertex `Attributes` bag with `Get_super_vertex_attributes` /
`Set_super_vertex_attributes`. Do not accept model existence or element-level
`Set_attribute(elt, ...)` as proof.
- **Reads may need compatibility fallback.** For source/sample data, read vertex-first and
fall back to element attributes only when old sample inputs were imported that way. Writes
of client output attributes stay vertex-only.
- **Inspector missing-vs-type logic:** use `Attribute_exists(Attributes, attr_name)` before
typed `Get_attribute(...)`. In 12d V15 this function returns non-zero when the attribute
exists; missing attributes inside an existing vertex bag must be counted as missing, not
type mismatches.
Local macro references to read when stuck
Abdullah explicitly pointed at these larger macros as reference material. Future agents
should inspect them before guessing widget, grid, file, source/TIN, event-loop, or
attribute patterns:
| Reference folder | Use it for |
|---|---|
| `C:\Users\a.almouhajer\OneDrive - Veris\BACKUP M6 USER FOLDERS\Documents\12d macros\_12D_MACRO_DATABASE\WIP_macros\Tin_Rename_Manager_Preview` | Small GridCtrl preview pattern, refresh/preview/commit button loop. |
| `C:\Users\a.almouhajer\OneDrive - Veris\BACKUP M6 USER FOLDERS\Documents\12d macros\_12D_MACRO_DATABASE\WIP_macros\Tin_Rename_Manager` | Larger editable GridCtrl readback: `Load_widgets_from_row`, per-column `Get_data`, apply-all loop. |
| `C:\Users\a.almouhajer\OneDrive - Veris\BACKUP M6 USER FOLDERS\Documents\12d macros\_12D_MACRO_DATABASE\WIP_macros\Tin_Rename_Manager_Bulk` | Bulk action UI and simple status/result patterns. |
| `C:\Users\a.almouhajer\OneDrive - Veris\BACKUP M6 USER FOLDERS\Documents\12d macros\_PLM\PLM_STR_Gauge\_Conform_Workflow\99_Handoff\PLM_KE_Clearance_R3_Handoff_20260515\01_Macros` | Proven `File_Box`, `Source_Box`, model/TIN/file path panels, file writing, long production event loops. |
| `C:\Users\a.almouhajer\OneDrive - Veris\0_IDEA\Veris_DNA\12d\macros\Veris_Label_Attributes_as_Text` | Attribute read/display patterns and Veris macro UI conventions. |
| `C:\Users\a.almouhajer\OneDrive - Veris\0_IDEA\Veris_DNA\12d\macros\Veris_Copy_Name_ID` | Attribute/name/ID copy patterns and compact Veris production macro structure. |
Known useful pattern from the PLM handoff macros: `File_Box x =
Create_file_box(label, status, CHECK_FILE..., "*.ext")`, then normal
`Set_data(x, path)` and `Get_data(x, path)`. Keep wildcard filters simple; the 12d
lesson says multi-extension wildcard strings are unsafe.
**Headless/agent testability (vision §1B):** when an auto-run flag / baked profile path is
supplied, skip `Wait_on_widgets`, pre-fill all handles from the sub-profile, and run — same
engine, no Run-button wait.
**POC (smallest proof, build when 12d is live):** a macro that reads a 3-field spec (1 text
input + 1 model/source reference + 1 TIN), builds those 3 widgets in a loop, lets the user
fill/select, saves the 3 values to a client sub-profile file, reloads + pre-fills them on a
second run. Prove variable-count widget storage + `Get_data` read-back + save/reload first;
only then scale to the full Vida spec.