Robotic surgery training programs: building evaluation rubrics for competency

A handful of sticky notes are still parked on my desk from last week’s skills lab—tiny rectangles filled with phrases like “camera drift,” “over-gripping,” and “slow clutching.” I used to think a good robotic training program was mostly about getting enough console hours. Now I think the heart of it is how we measure progress. Without a clear, fair rubric, feedback turns into vibes. With a rubric, the whole team—learners, faculty, and the hospital—can see the path from novice to safe, reliable practice.

How I stopped guessing and started scoring

My turning point came after a resident asked, “What exactly do I need to do so you can trust me with the console?” Trust is the operative word. In the U.S., competency-based education is finally becoming the default. The American Board of Surgery’s move toward Entrustable Professional Activities (EPAs) gives us a shared language for “ready for indirect supervision” and “needs direct guidance.” I keep their overview bookmarked for quick reality checks when I’m designing assessments—see ABS EPAs here.

• High-value takeaway: Design rubrics to answer one question: “Can I entrust this trainee to perform this task under this level of supervision, today?”
• Anchor your scale to a recognized framework (e.g., ABS EPAs for phases of care) so your rubric doesn’t live in a vacuum.
• Make scoring observable and behavior-based. If it can’t be seen or heard, it can’t be reliably assessed.

What to measure before anyone touches the console

Most “rubrics” fail because they start in the operating room. I’ve had better outcomes when I build pre-console checkpoints that capture preparation, safety, and systems awareness. I align these with the ACGME Surgery Milestones to keep expectations transparent and developmental over time (ACGME Milestones PDF is handy here and the supplemental guide is a great translator here).

• Case readiness — articulates indication, steps, and bailout plans; can list two meaningful intraoperative hazards.
• Platform safety — performs checklists for system setup, instrument integrity, and emergency undocking.
• Team brief — clarifies roles, signals for requesting help, and expected decision points.

These pre-op items are scored simply (0 = not demonstrated, 1 = partial, 2 = complete) and rolled into the overall entrustment decision for the day’s task. It keeps everyone honest about the “non-console” work that actually prevents harm.

The six core domains I use during console time

When the scope goes in and the robot wakes up, I switch to a more granular, validated structure. The Global Evaluative Assessment of Robotic Skills (GEARS) gives a solid backbone—depth perception, bimanual dexterity, efficiency, force sensitivity, robotic control, and (often adapted) autonomy. The original validation paper in Journal of Urology explains why these domains hold up across procedures; I link it whenever we’re calibrating faculty here.

• Depth perception — anticipates tissue planes; minimal “searching.”
• Bimanual dexterity — coordinated hands; avoids instrument collisions; smart use of third arm.
• Efficiency — purposeful movements; timely clutching; no wasteful camera travel.
• Force sensitivity — gentle retraction; atraumatic grasping; stops when tissue says “enough.”
• Robotic control — smart port placement; camera centered; ergonomic, stable console posture.
• Autonomy (entrustment lens) — requests help appropriately; completes defined steps with the agreed supervision level.

Scoring tip: I use a 1–5 anchor for each domain, where 1 = “unsafe without direct take-over,” 3 = “routine with prompted corrections,” and 5 = “consistently safe, anticipatory, efficient.” Each anchor is tied to examples we’ve written for our service lines, and those examples map back to EPA phases of care so the rubric doesn’t drift from broader training goals (ABS EPAs).

Nontechnical skills that change everything

Technical brilliance can’t rescue poor communication. I integrate nontechnical skill items into the same rubric—never as an afterthought. Borrowing the language of situational awareness, leadership, decision-making, and teamwork keeps it concrete and coachable.

• Shared mental model — confirms plan and “if/then” branches with bedside assistant and anesthesia.
• Workload management — slows or pauses when cognitive load spikes; verbalizes next step.
• Help-seeking — invites second opinions early; sets thresholds for escalation before trouble lands.

These items often explain why console performance swings from “great” to “shaky” on similar cases. They also fit cleanly into Milestones language for professionalism and systems-based practice (see ACGME Milestones here).

Making the numbers mean something

Scores don’t teach; feedback tied to scores does. I keep three rules on a laminated card:

• One behavior, one sentence — e.g., “Your camera horizon drifted after clutching, which hid the target plane.”
• One next step — “Try clutching after every third wrist roll and re-center your horizon to the vessel axis.”
• One resource — link a module or checklist to practice before the next case.

For foundational skills outside live OR time, a structured curriculum such as the Fundamentals of Robotic Surgery (FRS) provides proficiency targets and shared language for simulation tasks; the latest program information is consolidated here (note the 2025 transition of stewardship to SAGES).

Setting cut scores without breaking people

“What’s a passing score?” haunted me until I moved from a single pass/fail cut to tiered decisions that mirror entrustment:

• Tier A — Proceed independently with attending on standby (EPA: indirect supervision).
• Tier B — Proceed with proactive check-ins at predefined steps (EPA: conditional entrustment).
• Tier C — Proceed only with active coaching and take-over readiness (EPA: direct supervision).
• Tier D — Defer console time; complete simulation assignments first (FRS-style modules or targeted drills).

Tiers are set by a programmatic committee view, not a single score. We review the past three assessments, pre-op readiness, and nontechnical performance. That keeps one off-day from derailing progress and prevents “checklist tyranny.”

Collecting evidence across the whole pathway

I’ve stopped treating OR checklists, sim performance, and case logs as separate universes. They all feed the same question—“Can I entrust?” Here’s the minimal data set that has worked for us:

• Simulation proficiency — complete a short menu of FRS-aligned tasks with video evidence and time/accuracy thresholds (FRS).
• Case-linked EPAs — document entrustment decisions and supervision levels by phase of care for representative index operations (ABS EPAs).
• Milestones snapshots — semiannual CCC reviews that translate day-to-day observations into developmental trajectories (ACGME Milestones).

For urology and other specialties, society guidance also emphasizes structured curricula, VR simulation, and formal privileging pathways; the AUA’s robotic surgery SOP summarizes expectations succinctly here.

Common pitfalls I learned the hard way

• Too many items — a 40-line rubric guarantees rater fatigue. I aim for 6–10 console domains plus 3–4 nontechnical items.
• Vague anchors — “good” and “excellent” are not behaviors. Replace with observable examples (“re-centers camera every 2–3 instrument repositions”).
• Uncalibrated faculty — rater training is nonnegotiable. We watch two anonymized videos and score together every quarter.
• Binary thinking — use tiers and trendlines; people learn in loops, not ladders.

A simple starter rubric you can copy

Below is the skeleton I bring to new services. It’s short on purpose. We tailor examples to the specific operation after a few cases together.

• Pre-op readiness (0–2)
  • States indication, steps, bailout plans (0/1/2)
  • Safety checks: system, instruments, undocking (0/1/2)
  • Team brief with roles and thresholds (0/1/2)
• Console performance (1–5 each; GEARS-informed)
  • Depth perception (1–5)
  • Bimanual dexterity (1–5)
  • Efficiency (1–5)
  • Force sensitivity (1–5)
  • Robotic control (1–5)
  • Autonomy/entrustment for defined steps (1–5)
• Nontechnical skills (1–5 each)
  • Situational awareness (1–5)
  • Team communication (1–5)
  • Workload management (1–5)
• Entrustment tier today
  • A = indirect supervision; B = conditional; C = direct; D = defer + simulation

How we use it: We score in the room (quick anchors), then add one or two “next-step” comments per domain within 24 hours. Every third case, we review trendlines with the trainee and reset one practice target in the sim lab. For simulation tasks, I prefer proficiency-based benchmarks from FRS modules paired with time-limited, focused drills; it keeps practice purposeful and realistic (FRS overview).

Little habits I’m testing in real life

• Two-minute horizon reset — I pause every two minutes of console time to check camera horizon, depth, and third-arm tension. It cut our “where are we?” moments in half.
• Micro-objectives — before a case, I ask the learner for one micro-objective (“cleaner wrist angles while suturing”). The rubric comment must reference that objective.
• Calibration sprints — once a month we score two de-identified videos with the whole team to align anchors (10–12 minutes total).

Signals that tell me to slow down and double-check

• Repeated camera drift despite cues → switch roles, coach clutching cadence, or pivot to a simpler step.
• Force spikes (bleaching tissue, tearing) → revert to two-step dissection; narrate “force-light” techniques.
• Team confusion (missed calls, misaligned expectations) → re-brief and reset the plan before proceeding.

When these show up, I lean on program standards rather than gut calls. Milestones help frame “why we pause,” and EPA language keeps the decision fair and teachable (links again for easy access: ACGME Milestones, ABS EPAs).

Where I place society guidance in the mix

I’m cautious not to reinvent the wheel. For specialty-specific training (especially urology), I pull expectations from society guidance that calls for structured VR simulation, proctoring, and privileging pathways—see the AUA SOP summary here. It meshes cleanly with an FRS-style simulation on-ramp and GEARS-informed intraoperative scoring.

What I’m keeping and what I’m letting go

I’m keeping rubrics that are short, anchored to trusted frameworks, and paired with one actionable tip per domain. I’m letting go of vague praise, 30-item score sheets, and the myth that “more hours” guarantees competence. The sources below are the ones I revisit when I get stuck—their language keeps my evaluations fair, teachable, and defensible.

FAQ

1) What’s the difference between EPAs and Milestones in my rubric?
Answer: I use Milestones to describe the developmental arc over months/years and EPAs to make today’s entrustment decision for a specific task or phase of care. The ABS overview is a quick primer here.

2) Do I really need simulation before console time?
Answer: I’ve found that short, proficiency-based drills accelerate early learning and reduce unsafe trial-and-error in the OR. A consolidated FRS overview is available here.

3) Which technical domains are “must-haves”?
Answer: GEARS domains (depth perception, bimanual dexterity, efficiency, force sensitivity, robotic control, and autonomy) remain solid and adaptable across procedures; see the validation study here.

4) How do I keep faculty scoring consistent?
Answer: Quarterly rater calibration with shared videos works. I also put EPA supervision levels on the rubric so scorers must commit to a tier, not just a number (EPA overview here).

5) What about specialty-specific policies?
Answer: Societies often summarize expectations for training and privileging. For urology, the AUA’s robotic SOP is a good reference point here. Always adapt to your institution’s rules.

Sources & References

• ABS — Entrustable Professional Activities
• ACGME — Surgery Milestones 2.0
• J Urology (2012) — GEARS Validation Study
• AUA — Robotic Surgery SOP
• ISE/SAGES — Fundamentals of Robotic Surgery

This blog is a personal journal and for general information only. It is not a substitute for professional medical advice, diagnosis, or treatment, and it does not create a doctor–patient relationship. Always seek the advice of a licensed clinician for questions about your health. If you may be experiencing an emergency, call your local emergency number immediately (e.g., 911 [US], 119).