Fix parallel computing in as.sir.data.frame (#276)

* Fix parallel computing in as.sir.data.frame

Six bugs in parallel = TRUE mode:

1. PSOCK workers (Windows / R < 4.0) never had AMR loaded, so every
   exported/AMR function call failed. Added clusterEvalQ(cl, library(AMR))
   with a graceful fallback to sequential when the package cannot be loaded
   (e.g. dev-only load_all() environments).

2. clusterExport'd AMR_env was a frozen serialised copy; as.sir() on the
   worker wrote to AMR:::AMR_env while run_as_sir_column read from the stale
   copy, so the captured log was always wrong. Fixed by resolving AMR_env
   dynamically via get("AMR_env", envir = asNamespace("AMR")) inside the
   worker function, and removing AMR_env from clusterExport.

3. In the fork-based (mclapply) path each worker inherited the parent's full
   sir_interpretation_history. Capturing the whole log then combining across
   workers duplicated every pre-existing entry. Fixed by recording the log
   row count before the as.sir() call and slicing only the new rows
   afterwards.

4. run_as_sir_column used non-exported internals (%pm>%, pm_pull,
   as.sir.default) that are inaccessible on PSOCK workers after library(AMR).
   Replaced pipe chains with direct as.mic(as.character(x[, col, drop=TRUE]))
   and as.disk(...) calls, and changed as.sir.default() to as.sir() which
   dispatches correctly via S3.

5. With info = TRUE, worker forks printed per-column progress messages
   simultaneously, producing garbled interleaved console output. Per-column
   messages are now suppressed inside workers (effective_info = FALSE) while
   the outer "Running in parallel" / "DONE" messages still appear.

6. Malformed Unicode escape \u00a (3 hex digits) in the "DONE" banner was
   parsed by R as U+00AD (soft hyphen) + "ONE"; corrected to  .

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Add parallel computing tests to test-sir.R

Eight targeted tests verify correctness of the parallel as.sir() path:
identical SIR output vs sequential, matching log row counts, no
pre-existing history duplication, reproducibility across runs, results
consistency across max_cores values, single-column fallback, and no
per-column worker messages leaking when info = TRUE. All pass when only
1 core is available (parallel silently falls back to sequential).

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Fix as.sir() data.frame: preserve already-<sir> columns, exclude metadata

Issue #278: two related bugs in the column-detection / type-assignment pipeline.

Bug 1 – already-<sir> columns deleted on re-run
  Line 886 excluded already-sir columns from the type assignment (they
  stayed type "") causing the result loop to do x[,col] <- NULL, deleting
  them.  Fix: drop the !is.sir() guard so all untyped columns fall through
  to type "sir" and are re-processed correctly.

Bug 2 – metadata columns treated as antibiotics
  as.ab("patient") -> OXY, as.ab("ward") -> PRU.  The column detector
  accepted any column whose name matched an antibiotic code, regardless of
  content.  Fix: for name-matched columns that do not already carry an AMR
  class, also verify content looks like AMR data (all_valid_mics, all-
  numeric, or any SIR-like string).  all_valid_disks() is intentionally
  avoided here because it strips letters from strings (as.disk("Pt_1")==1).

Also adds tools/benchmark_parallel.R: a standalone script that times
sequential vs parallel as.sir() across n=20/200/2000/20000 rows and
saves a ggplot2 PNG to tools/benchmark_parallel.png.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Update benchmark: two-panel script with warm-up and column-count sweep

Previous single-panel benchmark was misleading: the first sequential run
paid one-time cache-warm-up cost (skewing n=20), and only 6 columns were
used so only 6 cores were ever active on a 16-core machine.

New two-panel design:
  Left  – vary rows with 16 fixed AB columns (shows memory-bandwidth
          saturation for large n)
  Right – vary columns with fixed rows (shows the real speedup profile:
          parallel wins when n_cols >> 1)

Also adds a warm-up pass before measurements to eliminate first-call bias.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Optimise parallel as.sir(): row-batch mode when n_cols < n_cores

Previously parallel dispatch only parallelised by column, so a 6-column
dataset on a 16-core machine used at most 6 cores with the other 10 idle.
For large n this also caused memory-bandwidth saturation (each worker did
a full n-row scan of clinical_breakpoints simultaneously).

New row-batch mode (fork path, R >= 4.0, non-Windows):
  pieces_per_col = ceil(n_cores / n_cols)
  Jobs = n_cols × pieces_per_col  (≈ n_cores jobs total)
  Each job: one column × one row slice

Benefits:
  - All cores stay busy regardless of column count
  - Per-worker memory footprint shrinks by pieces_per_col ×
  - Breakpoints lookup cache pressure reduced per worker

PSOCK path (Windows / R < 4.0) is unchanged: per-job serialisation
overhead makes row batching unprofitable there.

run_as_sir_column() gains an optional `rows` parameter (NULL = all rows,
backward-compatible). Results are reassembled via as.sir(c(as.character(.)))
which is safe for already-clean SIR values.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Fix info=FALSE ignored when no breakpoints found in as_sir_method

Operator-precedence bug at line 1601:

  if (isTRUE(info) && nrow(df_unique) < 10 || nrow(breakpoints) == 0)

R evaluates && before ||, so this was equivalent to:

  (isTRUE(info) && nrow(df_unique) < 10) || (nrow(breakpoints) == 0)

When nrow(breakpoints) == 0 (e.g. cefoxitin / flucloxacillin / mupirocin
against E. coli in EUCAST) the intro message was always printed regardless
of info. Fix: add parentheses so info gates both conditions:

  isTRUE(info) && (nrow(df_unique) < 10 || nrow(breakpoints) == 0)

Also pass print = isTRUE(info) to progress_ticker so the progress bar
(which prints intro_txt as its title) is suppressed when info = FALSE.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Fix cli formatting in as.sir() messages

- stop_if for empty ab_cols: wrap as.mic() and as.disk() in
  {.help [{.fun ...}](...)} for clickable links in cli output
- Parallel mode message: use {.field col} formatting for column names
  and quotes = FALSE in vector_and(), consistent with the rest of the
  codebase (avoids double-quoting from both font_bold and quotes="'")

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Use font_bold() inside {.field} for column names in parallel message

Convention: paste0("{.field ", font_bold(col), "}") gives bold green
column names without quotation marks, consistent with the rest of the
codebase (e.g. the 'Cleaning values' message in run_as_sir_column).

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Add collapse = NULL to font_bold() for column name vectors

font_bold() without collapse = NULL joins a vector with "" into a single
string, breaking paste0() element-wise formatting for length > 1 vectors.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Add tools/ to .Rbuildignore

Keeps the benchmark script out of the built package tarball.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

---------

Co-authored-by: Claude <noreply@anthropic.com>

tools/benchmark_parallel.R

@@ -0,0 +1,115 @@
+# Benchmark: sequential vs parallel as.sir() across data-set shapes
+#
+# Run from the repo root:
+#   Rscript tools/benchmark_parallel.R
+# or inside an R session:
+#   source("tools/benchmark_parallel.R")
+#
+# Two panels:
+#   Left  – fixed columns (n_ab_fixed), varying rows.
+#           Parallel wins at small n; sequential catches up at large n due to
+#           memory-bandwidth saturation (all workers compete for the same
+#           clinical_breakpoints lookup table in L3 cache / RAM).
+#   Right – fixed rows (n_rows_fixed), varying column count.
+#           This is the shape that actually benefits: each additional column
+#           keeps another core busy.  The "real world" gain for a 2854×65
+#           dataset lives here.
+#
+# Requires ggplot2; uses devtools::load_all() so the package need not be
+# installed.
+
+devtools::load_all(".", quiet = TRUE)
+
+# ── configuration ─────────────────────────────────────────────────────────────
+row_sizes   <- c(200, 1000, 5000, 20000)
+col_sizes   <- c(4, 8, 16, 32, 48)
+n_rows_fixed <- 1000
+n_ab_fixed   <- 16
+n_cores_avail <- AMR:::get_n_cores(Inf)
+
+all_abs <- c("AMC", "GEN", "CIP", "TZP", "IPM", "MEM",
+             "AMP", "TMP", "SXT", "NIT", "FOX", "CRO",
+             "FEP", "CAZ", "CTX", "TOB", "AMK", "ERY",
+             "AZM", "CLI", "VAN", "TEC", "RIF", "MTR",
+             "MFX", "LNZ", "TGC", "DOX", "FLC", "OXA",
+             "PEN", "CXM", "CZO", "KAN", "COL", "FOS",
+             "MUP", "TCY", "TEC", "IPM", "CHL", "FEP",
+             "MEM", "TZP", "GEN", "AMC", "AMX", "AMP")
+all_abs <- unique(all_abs)
+
+mic_vals <- c("0.25", "0.5", "1", "2", "4", "8", "16", "32")
+
+make_df <- function(n_rows, n_ab) {
+  set.seed(42)
+  ab_sel <- all_abs[seq_len(min(n_ab, length(all_abs)))]
+  mics <- lapply(ab_sel, function(a) as.mic(sample(mic_vals, n_rows, TRUE)))
+  names(mics) <- ab_sel
+  data.frame(mo = "B_ESCHR_COLI", mics, stringsAsFactors = FALSE)
+}
+
+time_both <- function(n_rows, n_ab, label) {
+  df <- make_df(n_rows, n_ab)
+  t_seq <- system.time(
+    suppressMessages(as.sir(df, col_mo = "mo", info = FALSE, parallel = FALSE))
+  )[["elapsed"]]
+  t_par <- system.time(
+    suppressMessages(as.sir(df, col_mo = "mo", info = FALSE, parallel = TRUE))
+  )[["elapsed"]]
+  message(sprintf("%-28s  seq=%5.2fs  par=%5.2fs  speedup=%.1fx",
+                  label, t_seq, t_par, t_seq / t_par))
+  data.frame(group = label, mode = c("sequential", "parallel"),
+             seconds = c(t_seq, t_par), stringsAsFactors = FALSE)
+}
+
+# ── warm-up (avoid first-call overhead biasing results) ───────────────────────
+message("Warming up cache ...")
+invisible(suppressMessages(as.sir(make_df(100, 6), col_mo = "mo", info = FALSE)))
+invisible(suppressMessages(as.sir(make_df(100, 6), col_mo = "mo", info = FALSE, parallel = TRUE)))
+sir_interpretation_history(clean = TRUE)
+
+# ── panel 1: vary rows, fixed columns ─────────────────────────────────────────
+message(sprintf("\nPanel 1 – varying rows, %d fixed columns:", n_ab_fixed))
+res_rows <- do.call(rbind, lapply(row_sizes, function(n) {
+  time_both(n, n_ab_fixed, sprintf("rows=%d", n))
+}))
+res_rows$x     <- rep(row_sizes, each = 2)
+res_rows$panel <- "Vary rows (16 fixed AB columns)"
+
+# ── panel 2: vary columns, fixed rows ─────────────────────────────────────────
+message(sprintf("\nPanel 2 – varying columns, %d fixed rows:", n_rows_fixed))
+res_cols <- do.call(rbind, lapply(col_sizes, function(n_ab) {
+  time_both(n_rows_fixed, n_ab, sprintf("cols=%d", n_ab))
+}))
+res_cols$x     <- rep(col_sizes, each = 2)
+res_cols$panel <- sprintf("Vary columns (%d fixed rows)", n_rows_fixed)
+
+results <- rbind(res_rows, res_cols)
+
+if (requireNamespace("ggplot2", quietly = TRUE)) {
+  p <- ggplot2::ggplot(
+    results,
+    ggplot2::aes(x = x, y = seconds, colour = mode, group = mode)
+  ) +
+    ggplot2::geom_line(linewidth = 1) +
+    ggplot2::geom_point(size = 2.5) +
+    ggplot2::facet_wrap(~panel, scales = "free_x") +
+    ggplot2::scale_colour_manual(
+      values = c(sequential = "#E05C5C", parallel = "#2E86AB")
+    ) +
+    ggplot2::labs(
+      title    = "as.sir() throughput: sequential vs parallel",
+      subtitle = sprintf("E. coli, EUCAST 2026, %d cores available", n_cores_avail),
+      x        = "Dataset dimension (rows  ·left·  or columns  ·right·)",
+      y        = "Wall-clock time (seconds)",
+      colour   = NULL
+    ) +
+    ggplot2::theme_minimal(base_size = 12) +
+    ggplot2::theme(legend.position = "top")
+
+  out_file <- "tools/benchmark_parallel.png"
+  ggplot2::ggsave(out_file, p, width = 10, height = 5, dpi = 150)
+  message("\nPlot saved to ", out_file)
+} else {
+  message("Install ggplot2 to get a plot; raw results:")
+  print(results[, c("panel", "group", "mode", "seconds")])
+}