Stats
Actions
Tags
From klayoutclaw
Registers source microscope images to a full_stack coordinate system using SIFT (same-substrate) or Chamfer+DE (cross-substrate) alignment for van der Waals stack detection.
How this skill is triggered — by the user, by Claude, or both
Slash command
/klayoutclaw:nanodevice_flakedetect_alignThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Register source microscope images to the full_stack target coordinate system.
Register source microscope images to the full_stack target coordinate system.
instrMCPdev with opencv, numpy, scipy, scikit-learn${PYTHON_PATH:-conda run -n instrMCPdev python} <script>Runs fully autonomously except for one mandatory pause: rotation selection after the coarse sweep.
1. Determine alignment type
├─ Same-substrate? → Run sift_align.py → DONE (if ≥20 inliers)
└─ Cross-substrate? → Continue to step 2
2. Run source_contour.py [--mirror]
→ View 01_source_contour.png: contour must trace the full flake boundary.
3. Run footprint.py [--mirror] [--bottom <bottom_part>]
→ Use --bottom when bottom_part image is available (diff mode, preferred).
→ *** CRITICAL: Verify footprint before proceeding ***
→ View 03_footprint_candidates.png — it shows multiple candidates side by side.
→ Compare EACH candidate against the source contour shape from step 2.
→ The default candidate (#1) is often WRONG — it may grab debris/satellite
flakes instead of the PDMS stamp. Candidates #2 or #3 are often better.
→ If 04_footprint_grabcut.png does NOT match the source flake shape:
Re-run with --candidate-rank 2 (or 3). Do NOT proceed with a bad footprint.
→ **Do NOT rely on shape_distance alone to pick candidates.** A candidate with
slightly worse shape_distance may produce much better IoU after sweep+refine
because the sweep optimizes position, rotation, AND scale. When all candidates
have shape_distance > 0.5 (none clearly good), run sweep+refine on at least
the top 2 candidates and compare final IoU.
4. Run sweep.py
→ Produces candidate overlay images
5. *** PAUSE: Select rotation ***
View 05_sweep_grid.png and individual candidate_NN.png files.
Pick the candidate where the contour best matches the flake.
IGNORE cost ranking — the lowest cost is often wrong.
6. Run refine.py --rot-hint <degrees>
→ **Runtime**: refine.py takes 10-15 minutes on 2-CPU machines (differential
evolution optimizer).
**MANDATORY EXECUTION METHOD**: Run refine.py as a FOREGROUND BLOCKING
command with a long timeout. Use the Bash tool with timeout=1200000
(20 minutes). Example:
Bash(command="${PYTHON_PATH:-conda run -n instrMCPdev python} .../refine.py ...", timeout=1200000)
Do NOT use run_in_background=true. Do NOT launch it as a background
process with &. Do NOT poll with sleep loops. Do NOT check for output
files in a loop. Just run the single blocking command and wait for it
to return. The Bash tool will hold until the process exits or the
timeout is reached.
→ Check metrics against acceptance thresholds (see below)
→ If accepted: DONE. warp_top.npy is ready.
→ If FAILED: Go to step 7.
7. *** RETRY LOOP (max 2 retries) ***
NEVER retry refine.py with the same footprint. Fix the INPUT first.
Retry 1: Re-run footprint.py with --candidate-rank 2
→ then sweep.py → select rotation → refine.py
Retry 2: Re-run footprint.py with --candidate-rank 3 or --n-clusters 24
→ then sweep.py → select rotation → refine.py
If still failing after 2 retries → STOP. Report failure.
IMPORTANT: Never retry refine.py more than once with the same footprint. If refine fails, the problem is the footprint or rotation selection, not refine's optimizer. Go back to step 3 and try a different
--candidate-rank.
Auto-accept when ALL pass:
| Metric | Pass | Borderline | Fail |
|---|---|---|---|
| fwd_chamfer_mean | < 2.5 um | 2.5-4.0 um | > 4.0 um |
| IoU | > 0.70 | 0.50-0.70 | < 0.50 |
| top_containment | > 0.90 | 0.80-0.90 | < 0.80 |
| outside_fraction | < 0.10 | 0.10-0.20 | > 0.20 |
Borderline: Accept but log a warning. Check diagnostic images. Fail on any metric: Do NOT accept. Adjust parameters and retry.
This is the core skill — reading diagnostic outputs and knowing which knob to turn.
Goal: The contour must capture the entire largest bright region — the full flake outline, including any very bright sub-regions (reflections, thin areas). A contour that misses the bright center but traces only the dim edges is wrong.
Common failure: Otsu auto-threshold can split the flake into "bright" and "very bright" regions, discarding the very bright part. In 01_source_contour.png, look for holes or missing chunks in the center of the flake — that means the threshold excluded the brightest pixels.
| What you see in 01_source_contour.png | What's wrong | Action |
|---|---|---|
| Contour traces the full flake boundary | Nothing | Proceed |
| Contour has a hole or missing center (very bright area excluded) | Otsu split the flake — bright part was thresholded out | Re-run with --gray-only to skip saturation threshold |
| Contour is too small / misses edges | Threshold too aggressive | Check if the image is very dark or low-contrast |
| Contour includes substrate/debris | Threshold too loose | Usually means the flake isn't the largest bright region — check source image quality |
| No contour found (area=0) | Flake not detected | Image may need manual inspection; verify it's the right file |
| What you see in diagnostics | What's wrong | Action |
|---|---|---|
| 04_footprint_grabcut.png matches source flake shape | Nothing | Proceed |
| Footprint grabs entire flake assembly + debris/satellite flakes | Default candidate (#1) picked up too much | Re-run with --candidate-rank 2 (or 3). Always check 03_footprint_candidates.png first — a better candidate likely exists |
| Footprint too large (includes bottom hBN) | Wrong clusters selected | Re-run with --n-clusters 20, --n-clusters 24 for finer segmentation |
| Footprint too small (misses edges) | GrabCut too aggressive | Re-run with --candidate-rank 2 or --candidate-rank 3 |
| Footprint is completely wrong shape | Shape matching failed | The source and target may look too different; check if --mirror is correct |
| shape_distance > 0.5 in stdout | Poor shape match | Continue anyway — GrabCut may still produce a usable footprint |
| What you see in candidates | Guidance |
|---|---|
| One candidate clearly matches | Use its rotation as --rot-hint |
| Two candidates look similar | Try the one where long edges align with visible flake edges |
| No candidate looks right | Footprint is likely wrong — go back to step 3 |
| Contour is right shape but shifted | Rotation is correct but translation is off — refine.py will fix this |
Key judgment: Look for edge alignment, not just overlap. The contour's straight edges should line up with the flake's crystallographic edges in the target image.
| Failed Metric | What it means | Adjustment |
|---|---|---|
| outside_fraction > 0.20 | Warped flake extends beyond footprint | Wrong rotation. Try the next-best sweep candidate. |
| IoU < 0.4 | Poor overlap between masks | Scale is wrong. Add --scale-hint with a value from the sweep candidate, ±0.1. |
| fwd_chamfer > 5 um | Contour edges don't align | Rotation off by a few degrees. Widen: re-run with --rot-hint ±5° from current. |
| top_containment < 0.80 | Much of warped flake is outside footprint | Footprint too small or rotation wrong. Check 21_mask_overlap.png: blue regions = warped-only = problem areas. |
| All metrics fail badly | Fundamentally wrong alignment | Start over. Re-examine footprint, try different rotation candidate, or check if --mirror is correct. |
Rule: NEVER retry refine.py with the same footprint. If refine fails, fix the footprint first. Time budget: Each refine.py attempt takes 10-15 min. Budget max 2 full attempts (footprint→sweep→refine). If 2 attempts fail and the best IoU is above 0.5, accept it and proceed — an imperfect alignment that lets you complete the pipeline is better than a perfect alignment that times out. Execution reminder: ALWAYS run refine.py as a foreground blocking Bash command with timeout=1200000. NEVER use run_in_background or sleep/poll loops.
Attempt 1: footprint (default) → sweep → select rotation → refine
→ If refine FAILS (IoU < 0.50):
Attempt 2: footprint --candidate-rank 2 → sweep → select rotation → refine
→ If still FAILS (IoU < 0.50):
Accept the best result from attempts 1-2 and proceed. Do NOT run a 3rd refine.
Max refine.py invocations: 2. Each takes 10-15 min — 3 would consume 45 min.
${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/sift_align.py \
--source <image> --target <image> --pixel-size <um/px> --output-dir <path> \
[--min-inliers 20] [--scalebar-bottom 0.08] [--scalebar-right 0.20]
Optional:
--min-inliers N — minimum RANSAC inliers for "sufficient" quality (default: 20). Thresholds: good ≥ max(50, 2N), warning ≥ N, insufficient < N. Lower to 10 for images with few substrate features.--scalebar-bottom F — fraction of image height to mask from bottom to exclude scalebar (default: 0.08). Set to 0 to disable.--scalebar-right F — fraction of image width to mask from right to exclude scalebar (default: 0.20). Set to 0 to disable.| Exit code | Meaning | Agent action |
|---|---|---|
| 0, ≥50 inliers | Good alignment | Done. Use warp_sift_bottom.npy |
| 0, ≥min-inliers | Marginal alignment | Accept with warning. Check 01_sift_matches.png |
| 2 | Too few matches (<min-inliers) | Try --min-inliers 10. If still fails, switch to Chamfer pipeline |
| 1 | Error | Check stderr |
Outputs: warp_sift_bottom.npy, 01_sift_matches.png, 01_sift_overlay.png (magenta-tinted warped source on desaturated target), updates alignment_report.json
${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/source_contour.py \
--image <image> [--mirror] [--gray-only] --output-dir <path>
Optional: --gray-only — use grayscale Otsu only, skip saturation intersection. Use this when the flake has very bright/overexposed areas that appear white (low saturation). Without this flag, bright areas are excluded by the saturation threshold.
Outputs: source_contour.npy, source_mask.png, 01_source_contour.png, updates alignment_report.json
SIFT-aligns bottom_part to target, computes LAB diff image, K-means on diff intensity. Isolates the top-placed flake from substrate. Splits disconnected blobs within clusters into sub-clusters before enumeration, so spatially separate flakes sharing the same intensity are treated independently.
${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/footprint.py \
--source <top_part> --target <full_stack_raw> \
--bottom <bottom_part> [--mirror] \
[--source-contour <out>/align/source_contour.npy] \
[--source-mask <out>/align/source_mask.png] \
--pixel-size <um/px> --output-dir <path>
Optional:
--source-contour + --source-mask — use pre-computed contour/mask from source_contour.py instead of re-segmenting internally. Recommended: ensures footprint uses the same source shape as sweep/refine.--n-clusters N — number of K-means clusters (default: 12; increase for finer segmentation on retry)--candidate-rank N — use the Nth-ranked candidate instead of the default (#1). Always check 03_footprint_candidates.png — candidate #1 is often wrong. Try --candidate-rank 2 or --candidate-rank 3 on retry.--warp <path-to-warp_sift_bottom.npy> — reuse the SIFT warp produced by sift_align.py instead of re-running SIFT internally (issue #31). If omitted, auto-resolves <output-dir>/warp_sift_bottom.npy then <source-parent>/../align/warp_sift_bottom.npy. Internal SIFT only runs when neither path exists. Sibling scripts (sweep.py, refine.py, source_contour.py) do not run SIFT internally and need no equivalent flag.Outputs: footprint_mask.png, footprint_contour.npy, 02_diff_image.png, 02_cluster_map.png, 03_footprint_candidates.png, 04_footprint_grabcut.png, updates alignment_report.json
${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/sweep.py \
--source-contour <.npy> --source-mask <.png> \
--footprint-contour <.npy> --footprint-mask <.png> \
--target-image <image> --pixel-size <um/px> --output-dir <path>
Outputs: candidate_01.png ... candidate_NN.png, 05_sweep_grid.png, updates alignment_report.json with "status": "needs_rotation_selection"
Auto re-sweep: If all top-8 candidates have scale < 0.75 (degenerate small-scale minimum), sweep.py automatically re-runs with scale floor raised to 0.75. This adds ~50s but avoids passing degenerate scales to refine.
${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/refine.py \
--source-contour <.npy> --source-mask <.png> \
--footprint-contour <.npy> --footprint-mask <.png> \
--target-image <image> \
--rot-hint <degrees> [--scale-hint <value>] \
--pixel-size <um/px> --output-dir <path>
Auto scale hint: When --scale-hint is omitted, refine.py reads alignment_report.json and uses the scale from the sweep candidate closest to --rot-hint. This constrains the search to ±0.1 around the sweep's estimate, avoiding the degenerate small-scale minimum.
Outputs: warp_top.npy, 20_best_overlay_raw.png, 21_mask_overlap.png, 22_chamfer_heatmap.png, updates alignment_report.json with "status": "complete"
warp_sift_bottom.npy: full_stack → bottom_part direction. Use cv2.invertAffineTransform() to go bottom_part → full_stack.warp_top.npy: source (top_part, possibly mirrored) → full_stack direction. Apply directly.npx claudepluginhub caidish/klayoutclaw --plugin klayoutclawAligns microscope stack images to a GDS fabrication template by detecting lithographic markers and computing image-to-GDS transform. Outputs warped image and material contours for KLayout.
Processes microscopy/bioimages with scikit-image: read/write, filter (Gaussian/median/LoG), segment (threshold/watershed/active contours), measure regions, detect features. SciPy/NumPy.
Extracts and preprocesses tiles from whole-slide images using histolab — tissue detection, tile extraction, and H&E stain normalization for pathology ML pipelines.