Data-only scene layout geometry — POST /v1/scene_geometry
Synchronous, free, read-only. Use this after /v1/video_data and before any screenshot / vision step when an agent needs to audit layout cheaply or pick coordinates for /v1/modify_scene.
/v1/scene_geometry resolves one scene by its stable voice_file, loads {voice_file}_spec.json if available, and returns narrator / caption / Remotion Storyboard display boxes in one 280×498 editor-preview coordinate space — plus dead zones and a structural summary. It never broadcasts MQTT, never renders screenshots, never exposes base64 assets, and never modifies the video.
It is designed for LLM-only agents that cannot see images: reason over JSON, then call /v1/modify_scene — typically with a layout.batch containing overlay.narrator.rect, overlay.caption.y, and remotion.object.rect.
🧭 Audit a scene by voice_file (preferred)
Default response — actionable data only. Cheapest, most reliable selector; paste the scene's voice_file from /v1/video_data.
Loading…
📝 Audit a scene by fuzzy narration text
Use when the user names a scene by what's said. Multi-match returns a ClarificationResponse — pick a voice_file and retry.
Loading…
🔬 Server/debug audit — include violations + warnings + counts
Opt back into the full debug payload (violations[], warnings[], summary.error_count/warning_count, remotion_poster_*). The mechanical collision math is approximate; treat results as hints.
Loading…
Recommended chain:
/v1/video_data→/v1/scene_geometry→/v1/modify_scene. Use/v1/scene_inspectoronly as an expensive last resort when you specifically need browser truth or a live screenshot.Mechanical linter output is debug-only. By default the response is actionable data only —
violations[](collision codes),warnings[](mobile-readability / face-staleness hints),summary.error_count/summary.warning_count, and the internalremotion_poster_*keys insideremotion_specare stripped. Vision-capable agents should judge collisions and aesthetics from a screenshot via/v1/scene_inspector; LLM-only agents should proceed with the rects +boxes.narrator.faceto keep overlays off the face. Passinclude_diagnostics: trueto opt back in for server/debug audits — see the Diagnostic mode section below.
Request
POST /v1/scene_geometry
Authorization: Bearer wc_live_REPLACE_ME
Content-Type: application/json
{
"id": "widecast7c0d4f8a9b1e2d3f",
"voice_file": "XcR0k"
}
Field reference
| Field | Type | Required | Description |
|---|---|---|---|
id |
string | yes | Topic id from /v1/video_data / /v1/create_video. Accepts widecast..., gubo..., or current ids. |
voice_file |
string | one of | Stable per-scene UID — preferred shortcut (server folds into by="voice_file"). |
scene_id |
string | number | one of | Current display/order id — fallback only. |
by |
string enum | one of | "voice_file" (preferred), "id", or "text". Required if neither voice_file nor scene_id is set. |
value |
string | number | one of | Match value for by. Required when by is set. |
min_score |
number | no | Only for by="text". Fuzzy-match threshold 0..1 (default 0.5). |
include_diagnostics |
boolean | no | Default false. When true, the response also carries violations[], warnings[], summary.error_count / summary.warning_count, and remotion_spec.remotion_poster_*. See Diagnostic mode. |
You must provide exactly one of: voice_file, scene_id, or by+value.
Response — 200 OK
{
"object": "scene_geometry",
"id": "widecast7c0d4f8a9b1e2d3f",
"topic_id": "widecast7c0d4f8a9b1e2d3f",
"company_id": "<company_id>",
"scene_id": 3,
"voice_file": "XcR0k",
"score": 1.0,
"coordinate_space": {
"width": 280,
"height": 498,
"unit": "editor_preview_px"
},
"safe_zones": {
"dead_top": { "x": 0, "y": 0, "w": 280, "h": 49.8, "coordinate_space": "preview", "reason": "top_10_percent_reserved" },
"dead_bottom": { "x": 0, "y": 373.5,"w": 280, "h": 124.5, "coordinate_space": "preview", "reason": "bottom_25_percent_reserved" },
"safe_rect": { "x": 0, "y": 49.8, "w": 280, "h": 323.7, "coordinate_space": "preview" }
},
"scene": {
"text": "Meta đã chi 14.3 tỷ đô để mua Scale AI…",
"talking_point": "AI training data is the new oil.",
"type": "STAT",
"pattern": "single_metric",
"keyword": "Scale AI",
"visual": "Chart of training data spend.",
"quote": "$14.3B",
"show_narrator": true,
"active_roll": "A"
},
"boxes": {
"narrator": {
"rect": { "x": 35, "y": 124, "w": 210, "h": 374, "coordinate_space": "preview" },
"visible": true,
"face_source": { "x": 30, "y": 60, "w": 100, "h": 100, "coordinate_space": "narrator_media_280x498" },
"face": { "x": 95, "y": 195, "w": 80, "h": 80, "coordinate_space": "preview" },
"face_center": { "x": 135, "y": 235 },
"face_source_coordinate_space": "narrator_media_280x498",
"face_coordinate_space_note": "face is converted from face_source through the current overlay.narrator rect"
},
"caption": {
"container_rect": { "x": 0, "y": 360, "w": 280, "h": 96, "coordinate_space": "preview" },
"text_rect_estimate": { "x": 10, "y": 370, "w": 260, "h": 60, "coordinate_space": "preview" },
"visible": true
},
"remotion": {
"groups": [
{ "id": "main", "rect": { "x": 0, "y": 0, "w": 280, "h": 498 }, "temporal_policy": "simultaneous_static" }
],
"objects": [
{ "id": "obj_01", "rect": { "x": 24, "y": 60, "w": 232, "h": 42 } },
{ "id": "obj_02", "rect": { "x": 24, "y": 110, "w": 232, "h": 42 } }
],
"object_layer": {
"objects": [
{
"layout_id": "main.one_by_one",
"rect": { "x": 24, "y": 140, "w": 232, "h": 42 },
"rect_canvas": { "x": 61.7, "y": 360.5, "w": 596.6, "h": 108.1 },
"temporal_policy": "one_by_one",
"update_field": "remotion.object.rect"
}
]
},
"sequence_objects": [
{ "id": "seq_03_text", "start": 0.0, "end": 2.5, "rect": { "x": 24, "y": 140, "w": 232, "h": 42 } }
]
}
},
"remotion_spec": {
"remotion_spec_file": "XcR0k_spec.json",
"remotion_spec_url": "https://widecast.ai/downloads/<company_id>/<topic_id>/XcR0k_spec.json?v=1748293012",
"remotion_spec_version": "1748293012",
"remotion_spec_exists": true,
"remotion_spec_state": "ready"
},
"summary": {
"remotion_object_count": 2,
"remotion_object_layer_count": 1,
"remotion_sequence_object_count": 1,
"remotion_group_count": 1,
"remotion_temporal_policies": ["simultaneous_static"],
"remotion_geometry_basis": "static_poster_objects_with_sequence_metadata"
},
"request_id": "req_abcd…"
}
Field reference
| Path | Description |
|---|---|
coordinate_space |
{width: 280, height: 498, unit: "editor_preview_px"}. All rect values use this space unless suffixed _canvas. |
safe_zones.dead_top |
Top 10% reserved (status bar / notch). |
safe_zones.dead_bottom |
Bottom 25% reserved (system gesture area). |
safe_zones.safe_rect |
Between the two dead zones. |
scene.* |
Narration + planning context: text, talking_point, type, pattern, keyword, visual, quote, show_narrator, active_roll. |
boxes.narrator.rect |
Current overlay.narrator.rect in preview coords. |
boxes.narrator.visible |
Narrator visibility flag. |
boxes.narrator.face_source |
Source-space face box from segment.narrator_face (narrator media coord system). Do NOT mutate segment.narrator_face for layout edits. |
boxes.narrator.face |
Displayed face box — converted from face_source through the current narrator rect. |
boxes.narrator.face_center |
Center point of the displayed face box. |
boxes.caption.container_rect |
Current overlay.caption rect in preview coords. Read-only for layout decisions — only overlay.caption.y is mutable. |
boxes.caption.text_rect_estimate |
Tight box around the rendered caption text (estimate). Use this to check collisions with the narrator face. |
boxes.remotion.groups[] |
Storyboard groups — {id, rect, temporal_policy}. Lower-level debug data. |
boxes.remotion.objects[] |
STATIC/POSTER display boxes resolved with the same Storyboard math used by {voice_file}_overlay_poster.png. Lower-level debug. |
boxes.remotion.object_layer.objects[] |
Agent-facing layer. Each item: layout_id, rect (preview), rect_canvas (Remotion 720×1280), temporal_policy, update_field: "remotion.object.rect". |
boxes.remotion.sequence_objects[] |
All temporal objects (every timed instance). Lower-level debug. |
remotion_spec |
Loaded {voice_file}_spec.json metadata: remotion_spec_file, cache-busted remotion_spec_url, remotion_spec_version, remotion_spec_exists, remotion_spec_state. Internal remotion_poster_* keys are diagnostic-only. |
summary |
Structural counts + policy summary (remotion_object_count, remotion_object_layer_count, remotion_sequence_object_count, remotion_group_count, remotion_temporal_policies, remotion_geometry_basis). error_count and warning_count are diagnostic-only. |
Diagnostic mode (include_diagnostics=true)
Pass include_diagnostics: true in the request body to opt back in to the full debug payload. This is intended for server / debug audits only — vision-capable agents should judge collisions and aesthetics from a screenshot via /v1/scene_inspector, and LLM-only agents should already have enough from the default response (rects + boxes.narrator.face) to keep overlays off the face.
POST /v1/scene_geometry
Authorization: Bearer wc_live_REPLACE_ME
Content-Type: application/json
{
"id": "widecast7c0d4f8a9b1e2d3f",
"voice_file": "XcR0k",
"include_diagnostics": true
}
The response adds these fields on top of the default body:
{
/* …default body… */
"violations": [
{
"code": "remotion_object_in_top_dead_zone",
"severity": "warning",
"a": "remotion.object_layer:main.one_by_one",
"b": "dead_zone.top",
"overlap": 0.18
}
],
"warnings": [
{
"code": "remotion_object_too_small_for_mobile",
"severity": "warning",
"object": "remotion.object_layer:main.obj_03_text",
"rect": { "x": 24, "y": 140, "w": 22, "h": 14 },
"message": "Object may be too small to read on mobile."
}
],
"summary": {
"error_count": 0,
"warning_count": 2,
"remotion_object_count": 2,
"remotion_object_layer_count": 1,
"remotion_sequence_object_count": 1,
"remotion_group_count": 1,
"remotion_temporal_policies": ["simultaneous_static"],
"remotion_geometry_basis": "static_poster_objects_with_sequence_metadata"
},
"remotion_spec": {
/* …same as default, plus: */
"remotion_poster_file": "XcR0k_overlay_poster.png",
"remotion_poster_url": "https://widecast.ai/downloads/<company_id>/<topic_id>/XcR0k_overlay_poster.png?v=1748293012",
"remotion_poster_version": "1748293012",
"remotion_poster_state": "ready",
"remotion_poster_exists": true,
"remotion_poster_warnings": []
}
}
Violation codes
| Code | Severity | Meaning |
|---|---|---|
remotion_object_overlaps_narrator_face |
error | Overlay object covers the narrator's face — agents must move it. |
remotion_object_overlaps_narrator_body |
info | Overlay object covers the narrator body (often desired). |
remotion_object_in_top_dead_zone |
warning | Overlay object intrudes into the top safe area. |
remotion_object_in_bottom_dead_zone |
warning | Overlay object intrudes into the bottom safe area. |
remotion_group_in_top_dead_zone |
warning / info | Whole Storyboard group intrudes top zone (info when group is the canvas wrapper). |
remotion_group_in_bottom_dead_zone |
warning / info | Whole Storyboard group intrudes bottom zone. |
caption_text_overlaps_narrator_face |
warning | Caption text estimate covers the narrator face. |
Warning codes
| Code | Meaning |
|---|---|
remotion_object_too_small_for_mobile |
Object width or height below mobile-readability minimum. |
narrator_face_outside_narrator |
Converted face center falls outside overlay.narrator.rect — narrator_face source data or rect may be stale. |
missing_narrator_face |
Scene has no narrator_face; face collision can only be approximated from the narrator body. |
The mechanical collision math is approximate. It treats every overlay object as a rectangle and the narrator face as a rectangle around the source face crop; staggered glyph corners, alpha-channel transparency, and animation timing aren't modelled. Treat these codes as hints, not ground truth — and prefer a real screenshot via
/v1/scene_inspectorwhen correctness matters.
Error responses
error.code |
HTTP | When |
|---|---|---|
missing_field |
400 | id is empty, or neither voice_file / scene_id / by+value is provided. |
invalid_id |
400 | id doesn't match an accepted topic id. |
invalid_by |
400 | by is not one of voice_file / id / text. |
video_not_found |
404 | No video with this id on the account. |
script_not_ready |
409 | Video is still processing — poll /v1/status/{id} first. |
script_parse_failed |
502 | Stored video_script is malformed. |
scene_not_found |
404 | The id / voice_file / text didn't match any scene. |
scene_geometry_failed |
500 | Internal error building the geometry — surface request_id and retry. |
missing_api_key / invalid_api_key |
401 | Auth. |
On ambiguous by="text" match the response is a 200 ClarificationResponse (no geometry) — same shape as /v1/modify_scene.
SDK examples
Python — audit + plan a layout.batch
from widecast import Widecast
client = Widecast()
geom = client.scene_geometry("widecast7c0d4f8a9b1e2d3f", voice_file="XcR0k")
face = geom["boxes"]["narrator"]["face"]
caption = geom["boxes"]["caption"]["text_rect_estimate"]
print("narrator face:", face)
print("caption text rect:", caption)
# Pick the agent-facing object-layer rect to move
target_obj = next(
o for o in geom["boxes"]["remotion"]["object_layer"]["objects"]
if o["layout_id"].endswith(".one_by_one")
)
print("layout_id:", target_obj["layout_id"], "rect:", target_obj["rect"])
# Push the overlay below the face center
new_y = max(int(face["y"]) + int(face["h"]) + 8, 200)
result = client.modify_scene(
"widecast7c0d4f8a9b1e2d3f",
by="voice_file", value="XcR0k",
fields=[
{"field_name": "remotion.object.rect",
"value": {"layout_id": target_obj["layout_id"], "y": new_y,
"coordinate_space": "preview"}},
],
)
print("new spec:", result["remotion_spec_url"])
TypeScript — collision audit
import Widecast from "@widecast/sdk";
const client = new Widecast();
const geom = await client.scene_geometry("widecast7c0d4f8a9b1e2d3f", {
voice_file: "XcR0k",
});
const errors = geom.violations.filter((v: any) => v.severity === "error");
if (errors.length > 0) {
console.log("Layout problems:");
for (const v of errors) console.log(" ", v.code, "→", v.a, "vs", v.b);
}
Manual (curl)
curl -sS -X POST ".../v1/scene_geometry" \
-H "Authorization: Bearer wc_live_REPLACE_ME" \
-H "Content-Type: application/json" \
-d '{ "id": "widecast7c0d4f8a9b1e2d3f", "voice_file": "XcR0k" }' | jq
When to use which surface
| Question | Tool |
|---|---|
What scenes are in this video? What's the voice_file of scene 3? |
/v1/video_data |
| Where is the narrator face / caption / overlay object on this scene? | /v1/scene_geometry |
| What does the editor actually look like right now in the user's browser? | /v1/scene_inspector (expensive) |
| Move / resize an overlay object | /v1/modify_scene remotion.object.rect |
| Move the narrator + caption + overlay in one shot | /v1/modify_scene layout.batch |