v0.3.0: OCR via camera, FAB UX, side drawer #4

Merged
jpnadas merged 16 commits from feature/ocr-camera into main 2026-06-09 18:38:34 +00:00
Owner

Release notes in CHANGELOG.md

Release notes in CHANGELOG.md
jpnadas added 16 commits 2026-06-09 18:38:24 +00:00
New "Scan image" button in the input pane header:

- Opens the phone's rear camera (capture=environment) or a desktop
  file picker via a hidden <input type="file">.
- On image selected, lazy-imports tesseract.js from esm.sh — no
  bundling, no Docker image growth.
- Language follows the "From" dropdown (ISO 639-2 mapping for the
  30 currently-supported languages). "Auto-detect" loads the
  eng+por+deu+nld pack and lets tesseract pick.
- Progress status renders below the input (engine load → OCR
  percentage → "done in Xs").
- Result replaces the editor contents; existing vim bindings,
  Shift-H/L nav, Ctrl-Enter translate all keep working.

First scan triggers a ~5 MB Tesseract WASM download plus the chosen
language pack (~3-10 MB). Cached by the browser after.
With the radagast server flipped to OLLAMA_KEEP_ALIVE=-1, /api/ps
returns expires_at as year 4001 (effectively forever), which the
old code rendered as "Hot: gemma2:9b — unloads in 17500000h". Add
a 7-day threshold to formatRemaining(); anything beyond that just
shows the model name with no expiry suffix.
Phone cameras emit 12+ MP images (~4000 px long edge). Tesseract
on a full-resolution shot was painfully slow — minutes, not
seconds. Downscaling to 1600 px on the long edge before
worker.recognize() makes OCR 4-8× faster with no real accuracy
loss for printed text.

Implementation: createImageBitmap -> draw to a sized HTMLCanvas
-> canvas.toBlob() at JPEG 0.85. Smaller-than-1600 px inputs are
passed through unchanged. Status line briefly shows the
downscale ratio so the timing is interpretable.
Two layers to stop tesseract hallucinating text on phone photos:

1. Bradley adaptive threshold (O(N) via integral images) turns the
   downscaled grayscale image into a binary B/W mask. Cuts out
   gradients, shadows, and surface texture that tesseract
   misreads as ink.
2. After recognise(), filter data.words by confidence >= 60 and
   re-assemble lines from kept words. Status line surfaces how
   many were dropped so the cost is visible.

Output of preprocess stage is PNG (not JPEG) so the sharp B/W
edges don't get re-blurred.
After picking an image, a modal opens with cropperjs's drag-handle
crop UI and Rotate left / Rotate right / Reset buttons. Cropping
to just the text region is the single biggest quality win for
OCR on phone photos — strips out logos, lighting transitions, and
background junk that bleed past the confidence filter.

- Lazy-loads cropperjs + its CSS from esm.sh on first use (no
  bundling; same pattern as Tesseract.js and CodeMirror).
- Honours EXIF orientation from phone shots (checkOrientation).
- Keyboard: Esc cancels, Enter accepts. Clicking the backdrop
  also cancels.
- Output goes through the existing preprocessForOCR pipeline
  (downscale to 1600px + Bradley adaptive threshold) before
  Tesseract sees it.
- Scan button is now a 📷 floating action button at bottom-right
  of the viewport (camera emoji per explicit request). Replaces
  the small "Scan image" button in the input header.
- Crop modal: inner padding on .modal-body so cropperjs's
  bottom-right corner handle isn't on top of Android's edge-swipe
  back-gesture zone. Backdrop also respects iOS safe-area insets
  (env(safe-area-inset-{right,left,bottom})).
- Material Symbols Outlined webfont (Google Fonts) — used for the
  FAB camera (photo_camera), and the rotate_left / rotate_right /
  restart_alt / check buttons in the crop modal. Replaces the
  platform-rendered 📷 emoji that was off-centre, and replaces
  text labels on rotate/use buttons. Cancel stays text.
- Cropper: autoCrop=false — no default selection. User drags on
  the image to draw the crop region; if they "Use this" without
  drawing, falls back to the full (possibly rotated) image.
- Modal-body padding bumped from 1.25 rem to 2 rem so the
  bottom-right corner handle sits ~48 px from the screen edge,
  outside Android's edge-swipe back-gesture zone.
- Header copy updated to "Drag on the image to select the text
  region" since there's no longer a default box to convey intent.
Four wins:

1. Replace <input type=file capture> with a getUserMedia
   viewfinder modal on mobile. Tapping the shutter goes straight
   to the crop modal — skips the OS "retake/use" confirm step.
   Falls back to file picker if camera permission is denied or
   the API is unavailable. "Pick from file" icon
   (photo_library) inside the camera modal opens the gallery
   on demand.

2. Drop the Reset button — with no default crop box, "reset" is
   just "no box again," which is the starting state anyway.

3. The check (Use this) button is now larger and more prominent
   — wider padding, 24px icon, accent fill.

4. Invert-colours toggle (invert_colors icon) in the crop modal.
   White-on-dark text (signs, menus, screenshots) breaks
   Tesseract because the binariser was made for ink on paper.
   Toggling the button flips the grayscale buffer before
   adaptive thresholding so the binary output ends up
   black-on-white. State plumbed cleanly: openCropper now
   resolves with {blob, invert}; preprocessForOCR takes an
   {invert} option; binarizeAdaptive applies the flip.

Also: drop `capture=environment` from the file <input>. Without
it the file path correctly opens the gallery/file dialog instead
of re-opening the OS camera when the user picks "Pick from file".
@media (max-width: 640px):
- backdrop loses its 1 rem inset; modal goes 100vw × 100vh with
  no border-radius
- crop modal body keeps ~20 px horizontal padding so cropper
  corner handles stay safely outside Android's edge-swipe
  back-gesture zone (the corner DRAG is what triggers it; a tap
  on a button at the edge doesn't)
- camera viewfinder body has zero padding — video fills the
  whole space
- footer has minimal 0.5 rem padding so the rotate / invert /
  cancel / check buttons hug the screen edges
- iOS notch / home-indicator insets honoured via
  env(safe-area-inset-*)

Doesn't change the camera-confirm behaviour — that's the OS-camera
fallback path that fires when getUserMedia is denied for not being
a secure context (i.e. http://lan-ip:8000 from the phone, not
HTTPS or localhost).
Previous attempt was actually-broken — typo notwithstanding, the
mix of 100vh, no explicit flex-shrink, and inconsistent body
padding was easy for the body to grow and shove the footer
buttons offscreen.

This pass:
- 100dvh instead of 100vh (mobile chrome / address bar collapses
  don't cause overflow / under-shoot)
- flex-shrink: 0 on .modal-header and .modal-footer so the body
  can never squeeze them out
- explicit background on the footer so it's not transparent
  against whatever's behind
- camera <video> fills modal-body with object-fit: contain
- body padding kept just enough (~16 px) for crop handle safety
Promotes Translate from an inline button under the input pane to
the primary 56x56 FAB at the bottom-right corner. Camera FAB
becomes the secondary in the stack, just above. Both share a
single .fab base class; positioning via .fab-primary /
.fab-secondary.

- Translate icon: Material Symbols `translate` (28 px).
- Pulsing-ring `busy` animation while a translation runs, since
  the icon-only button can't show "Translating…" text anymore.
- body padding-bottom bumped to ~9 rem so the output pane
  scrolls clear of the stacked FABs.
- ⌘/Ctrl-Enter still triggers translate via the existing
  CodeMirror keymap; tooltip text updated.
Two-pronged fix so the Translate + Scan FABs stay reachable when
the mobile soft keyboard is open:

- Add `interactive-widget=resizes-content` to the viewport meta —
  Chrome on Android then shrinks the layout viewport when the
  keyboard opens, so `position: fixed; bottom: ...` is naturally
  above it. iOS Safari ignores this directive, hence:
- VisualViewport listener computes
    inset = innerHeight - vv.height - vv.offsetTop
  and writes it to a `--kb-inset` CSS custom property. The FAB
  bottom calc uses `max(var(--kb-inset), safe-area-inset-bottom)`
  so the keyboard inset wins when it's open and the home-indicator
  inset wins when it isn't.

Net: tap the editor on iOS → keyboard slides up → Translate /
Scan FABs rise with it, no manual minimise required.
- Controls go 3 cols → 2 cols (From / To). Model selector moves
  into a kebab (⋮ more_vert) menu in the top-left corner, using
  a native <details> element so toggle and a11y come for free.
- Default source language: Dutch (was Auto-detect). Target stays
  English. Quickest path to a translation since the typical use
  case is reading Dutch text.
- Drop the VisualViewport listener and --kb-inset CSS var —
  Android Chrome already handles keyboard with the
  interactive-widget=resizes-content meta directive; iOS Safari
  isn't in scope. FAB bottom calc reverts to just safe-area-inset.
Replace the <details>-based kebab popover with a proper Material-
style navigation drawer:

- Hamburger button (menu icon) at top-left.
- <aside> drawer slides in from the left via transform: translateX().
  width: min(85vw, 320px). Backdrop overlay behind it.
- Close paths: X button in the drawer header, backdrop click, Esc.
- Padding honours Android safe-area insets on all four sides.
- Model selector + label live inside the drawer; future per-app
  settings can move here too.
The h1 + tagline at the top of the main view were just taking
vertical space. Strip them. The "Finrod" wordmark now lives only
inside the side drawer, set in Cinzel (classical-inscription
serif, loaded from Google Fonts) alongside the Star of Fëanor
mark — matches the favicon and the loremaster vibe.

Body padding-top adjusted to 3.5 rem so the controls clear the
fixed hamburger button at the top-left.
jpnadas merged commit fd8f0b74aa into main 2026-06-09 18:38:34 +00:00
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: jpnadas/finrod#4
No description provided.