Commit Graph
561 Commits
Author SHA1 Message Date
dr. M.S. (Matthijs) Berends 08aa47842e (v3.0.1.9152) Let a named colours_SIR without S, I or R use the default colours for these, which fixes the example of ggplot_sir() with colours = c(SI = "yellow"), let ggplot_sir() pass the facet as a value to facet_sir(), and make the test of the eucast_I default independent of the AMR_guideline option 2026-10-16 04:34:52 +02:00
dr. M.S. (Matthijs) Berends 43accd35d1 (v3.0.1.9151) Fix inconsistencies in the plotting functions and correct the translations of the interpretation categories
Plotting:
- Base colours and legends of interpreted MIC and disk values on the interpretation itself instead of matching colours
- Fix SDD in autoplot() for disks, the colours of barplot() for SIR values, and barplot() for MICs ignoring language and expand
- Match a named colours_SIR by name; let keep_operators = TRUE/FALSE in scale_*_mic() behave as "all"/"none"
- Let a named values in scale_*_sir() replace the colours it names, warn in scale_x_sir(), and pass other arguments on
- Let the default of eucast_I follow the guideline option, also for EUCAST before 2019
- Support unquoted column names in facet_sir() without evaluating them
- Correct the documentation and examples

Translations:
- Add SDD, NS, WT and NWT, and correct S, I, R and NI in 18 languages
- Sources of the terms for 'I': NAK via German laboratories (de), SZU national reference laboratory (cs), lex.dk (da), Karolinska University Hospital (sv), COESANT (es), Nordlab (fi), CA-SFM (fr); no by analogy with da and sv
- Not verified against an official source: ar, bn, el, hi, id, ja, ko, pl, ro, ru, sw, tr, uk, ur, vi, zh

Tests:
- Let the grouped antibiogram test use English so it passes in other locales
2026-10-15 22:25:20 +02:00
dr. M.S. (Matthijs) Berends eb9a9c2fa5 (v3.0.1.9149) Fix WHONET codes in microorganisms.codes that referred to unrelated microorganisms, let codes refer to the name they were named after and add a source column, and replace the fuzzy matching of WHONET names by exact matching with a guard against unreviewed remaps
WHONET names are now matched exactly (also outdated names, after removing qualifiers), then on GBIF identifier, and fuzzy results are only kept if consistent with that; otherwise the genus or UNKNOWN is used. A code refers to the record of the name it was named after (e.g. EAE is Enterobacter aerogenes) if that name refers to the same taxon. Every remap of an existing code to another taxon must be listed in data-raw/whonet_remaps_reviewed.csv; the 198 remaps of this rebuild were reviewed.
2026-10-15 21:08:04 +02:00
dr. M.S. (Matthijs) Berends 179eec584f (v3.0.1.9148) Store zone diameters without S category as blocking rows for EUCAST and as R-only rows for CLSI instead of capping them at 50 mm, let as.mo() return the complex for names such as 'Acinetobacter calcoaceticus-baumannii complex', make the breakpoint reproduction scripts stop on unchecked data defects and year-independent, and document that as.disk() returns NA above 50 mm
The full rebuild of clinical_breakpoints also assigns Salmonella serovars and the Bacteroides fragilis complex correctly and adds ECOFFs for Neocosmospora solani. The rebuilt microorganisms.codes is not included, since it remaps WHONET codes that need review first.
2026-10-15 17:05:41 +02:00
dr. M.S. (Matthijs) Berends fcfaa87f5d (v3.0.1.9147) Keep EUCAST blocking rows for infections other than UTI where EUCAST only gives a UTI breakpoint, e.g. breakpoints in brackets for aminoglycosides in Acinetobacter, and let as.sir() with unknown uti consistently prefer non-UTI breakpoints, use UTI breakpoints only if no others exist and note this in sir_interpretation_history() every time
The pruning of blocking rows in the EUCAST reproduction script now compares UTI status too, and the script is fixed for the renamed column of the as.ab() coercion cache. In clinical_breakpoints, only the blocking rows were replaced (356 organism/agent/method combinations newly blocked, none unblocked); other changes of a full rebuild, caused by taxonomy updates in the WHONET-based rows, are not included.
2026-10-15 14:55:57 +02:00
dr. M.S. (Matthijs) Berends b3ecc01657 Show translated pathogen names instead of microorganism codes in grouped traditional antibiograms, including for unknown species 2026-10-15 10:31:07 +02:00
dr. M.S. (Matthijs) Berends eb6d07e1fe Include isolates of unknown species in grouped antibiograms and in wisca(), which were silently excluded with the warning 'invalid microorganism code, NA generated' 2026-10-15 10:21:44 +02:00
dr. M.S. (Matthijs) Berends 0efc8826c8 Let aggregated analyses follow the guideline for the 'I' category in count_df(), proportion_df(), sir_df(), the plotting functions, antibiogram(), wisca(), and bug_drug_combinations(), deprecate combine_SI in favour of guideline, fix format() on bug_drug_combinations() counting I as resistant, and add a shared section 'Interpretation of SIR' 2026-10-15 09:58:21 +02:00
dr. M.S. (Matthijs) Berends dbd95ffa46 (v3.0.1.9137) Add the guideline argument to count_df(), proportion_df(), sir_df(), ggplot_sir(), geom_sir(), and labels_sir_count(), so that with CLSI the 'I' category is merged with R, and describe in the documentation of get_AMR_locale() that LANG takes precedence over LANGUAGE 2026-10-15 08:25:37 +02:00
dr. M.S. (Matthijs) Berends 46f5de1d36 (v3.0.1.9136) Fix the CI entry with a Dutch locale by using a UTF-8 locale that is installed on the runner, and skip the mo_source tests with skip_if() so that the tinytest harness for old R versions supports them 2026-10-15 00:11:07 +02:00
dr. M.S. (Matthijs) Berends 8b01bcb23c (v3.0.1.9135) Fix get_mo_source() removing the mo_source file at the default location, returning the wrong mo_source, and throwing an error after the original file was moved, and run the unit tests in CI also in a real interactive session and with a user setup, a Dutch locale, and CLSI as default guideline, with tests that no longer assume an English locale or EUCAST 2026-10-14 23:34:45 +02:00
dr. M.S. (Matthijs) Berends 4be7352843 (v3.0.1.9134) Add session cache assertions for separate as.ab() and as.mo() calls, for get_mo_source() without a file, and for real cache hits, which detect the cache being emptied in interactive sessions 2026-10-14 22:49:00 +02:00
dr. M.S. (Matthijs) Berends 1a1a3abed1 (v3.0.1.9132) Make translating around 50 times faster and coercion of unknown input in as.ab() up to 70 times faster with identical results, use DOIs instead of PubMed URLs, shorten an example line, and reword cran-comments.md 2026-10-14 20:17:28 +02:00
dr. M.S. (Matthijs) Berends 9ebc33a59d (v3.0.1.9129) Add guideline argument to random_mic() and random_disk() so that values are based on one guideline version, make guideline years in examples, vignettes and website always the latest, and reword NEWS 2026-10-13 21:43:19 +02:00
dr. M.S. (Matthijs) Berends 04a2f0b3ac (v3.0.1.9125) Fix translations being translated again (e.g. Ticarcillinee, groupee) and untranslated parts of combination names, and let as.mo() recognise translated names in the set language, cached per language 2026-10-12 22:25:57 +02:00
dr. M.S. (Matthijs) Berends d4ba5051e7 (v3.0.1.9124) Update French CoNS name in translation test after improved French translations (#322) 2026-10-12 21:39:36 +02:00
dr. M.S. (Matthijs) Berends bf12e3abe7 (v3.0.1.9123) Add method argument to sir_combine() for resistance within a group (most_resistant), and restore the AMR 3.0.1 behaviour of count_*() and proportion_*() for multiple antimicrobials (#321) 2026-10-12 21:37:13 +02:00
dr. M.S. (Matthijs) Berends 10053d582e (v3.0.1.9117) Fix static test in test-zzz.R to run without testthat (R 3.0/4.0 tinytest) 2026-10-11 22:15:43 +02:00
dr. M.S. (Matthijs) Berends 53d0704271 (v3.0.1.9116) New sir_combine() for combination therapy as one engine for antibiogram(), wisca(), count_*() and proportion_*() (#321), and make all suggested packages runtime-only with a new CI workflow without dplyr
- New sir_combine() to combine SIR results of multiple antimicrobials into one variable; antibiogram(), wisca() and the count/proportion functions now use the same engine (antibiogram() and wisca() output unchanged)
- With multiple antimicrobials, count_*() and proportion_*() now follow the documented combination therapy rules: resistance() is the proportion of failing combinations, count_all() excludes isolates with an unknown combination result, and WT/NWT/NS/NI are handled correctly
- where() is now decided at runtime instead of at installation, fixing antibiogram() and wisca() without tidyselect (e.g. CRAN binaries without dplyr)
- Imports from data.table and cli moved to .onLoad(); data.table functions are only used while it is loaded; the like() example no longer unloads data.table, which crashed R
- Static test that fails on code outside functions depending on suggested packages
- New check-without-dplyr workflow: installs AMR with all suggested packages, removes dplyr (and tidyselect), then runs all unit tests and examples against the installed package
- g.test() now returns the degrees of freedom as 'parameter', so they are printed
- Fix R CMD check NOTEs (hidden .claude folder, lost braces in inherited g.test and ggplot_pca docs)
2026-10-11 20:36:31 +02:00
dr. M.S. (Matthijs) Berends 80afedeb8c (v3.0.1.9115) Fix interpretation of capped MICs without an S category (off-scale S breakpoint) or without breakpoints in as.sir(), and correct the documented default of capped_mic_handling (#295) 2026-10-11 15:40:09 +02:00
dr. M.S. (Matthijs) Berends 9e609dae83 (v3.0.1.9114) Fix errors without cli and on R 3.0, and make the tinytest pipeline actually run the unit tests
Package:
- stop_() called cli::ansi_has_hyperlink_support() before checking that cli is available, so without cli every error became "there is no package called 'cli'"
- interpretive_rules() failed on R 3.0 for guidelines without AmpC rules (assignment to zero rows)
- Guard the ggplot2 example of wisca_plot()

CI:
- check-old: R 3.0 could not install tinytest over HTTPS, so no unit tests ran; install it with curl, and fail when tinytest is missing or any test fails
- tinytest shim: record expectations in the test file environment, support skips and errors per test_that() block, add missing testthat equivalents, source helper files
- Add R 4.0 (oldrel-6) to check-old, and oldrel-5 to check-recent
2026-10-10 14:54:15 +02:00
dr. M.S. (Matthijs) Berends f0699c6584 (v3.0.1.9113) Fix infinite loop in the plain-text message fallback (used without cli or with old cli/rlang), e.g. for wisca(parallel = TRUE) without a future plan 2026-10-10 13:27:54 +02:00
dr. M.S. (Matthijs) Berends 16a13ba81f (v3.0.1.9109) Fix WHONET codes SUD and ROX in clinical_breakpoints, and add sulbactam/durlobactam (#308) 2026-10-10 09:42:25 +02:00
tori0513 3eeabf542a Add add_custom_antimicrobial_synonyms() (#307) (#311)
Adds add_custom_antimicrobial_synonyms() to add local names, such as trade names, to existing antimicrobials, also via the AMR_custom_ab option. Matching is language-independent; as.ab() now returns NA with a warning for input that is not valid UTF-8.
2026-10-10 09:14:07 +02:00
dr. M.S. (Matthijs) Berends 9e05fe184b (v3.0.1.9106) Fix trade names of combinations filed under a single agent, and remove non-synonyms from antimicrobials (#315) 2026-10-09 22:25:44 +02:00
Matthijs BerendsandClaude Opus 5.5 891d248fab Improve the session caches of as.ab(), as.mo() and all mo_*() functions (#313) (#314)
- One cache design for as.ab() and as.mo() in R/aa_session_cache.R: lazily built
  lookup indices (ATC codes, drug synonyms, SNOMED codes) and coercion caches with
  the same structure (key, input, value), keyed on input plus outcome-relevant settings
- Caches are invalidated whenever the lookup tables change (custom antimicrobials or
  microorganisms, set_mo_source())
- mo_*() functions use a fast path for valid <mo> input and work on unique values only
- synonym_mo_to_accepted_mo() is memoised per code
- Fix as.mo() cache keys that never matched again, fast_mode results of as.ab() being
  reused by regular calls, and ab_reset_session() clearing as.mo() uncertainties

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:29:48 +02:00
Matthijs BerendsandClaude Opus 5.5 0ca43df459 Update taxonomy of microorganisms (2026-10) (#312)
* Update taxonomy of microorganisms (2026-10)

Rebuild of the microorganisms data set from COL 2026-04-18 XR, LPSN and BacDive of October 2026
and MycoBank of January 2026, with v3.0.1 as the previous data set: 111,103 records (v3.0.1:
78,679), 0 changed MO codes of existing names, 522 released codes retired (275 to be reviewed).

Script fixes (data-raw/_reproduction_scripts/reproduction_of_microorganisms.R):
- previous data set from the last release; COL read as COL Data Package (NameUsage.tsv)
- keep COL records without authors (a filter on NA dropped them, e.g. Balamuthia mandrillaris)
- genus homonyms: unclassified MycoBank entries are not authoritative; the released domain of a
  genus wins if a source supports it (e.g. Moorella, Serpula, Pirella stay Bacteria)
- protect current names of relevant species instead of whole current genera (data set tripled)
- current names and synonyms only within the same domain and for known names (Brugia -> Lycoperdon)
- MycoBank names in Latin script only; old records outside their source's scope become manually added
- Graphium butterflies of v3.0.1 are not carried over, re-added or restored
- restore released taxa by name and rank; retire older cross-domain homonyms for review
- fill empty families from COL and a commented override list (Ascaris, Balamuthia)
- add Brugia malayi and B. timori; remove Salmonella species synonyms that are serovars
- BacDive: a minority of facultative strain records no longer decides (C. difficile)
- translate codes of other data sets with the development version; review unmatched codes
- scope decisions: COL XR names only if relevant or released; Protozoa follow the Fungi rules

Also: all checkpoints (data-raw/taxonomy*.rds), updated data sets, Bartlett et al. list, accessed
dates in TAXONOMY_VERSION, NEWS, two staphylococci added to CoNS, tests updated for real taxonomic
changes, known defects file removed (all solved), and the findings report with all review files in
data-raw/taxonomy_review/2026-10/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: note on the .dta export and the single commit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix as.mo() for Salmonella serovars written with the species; add S. xeri to CoNS

- "Salmonella enterica (subsp. enterica) serovar Typhi" returned S. bongori, and
  "S. enterica Typhimurium" S. enterica diarizonae; "Paratyphi A/B/C" became serogroups
- Staphylococcus xeri and S. parequorum confirmed as coagulase-negative

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy build: remove lichens (decision of 5 October 2026)

Fungi in the lichen-forming classes and the order Verrucariales are removed unless their genus is
clinically relevant, with the synonyms that point to them (4,155 records); released lichens are
restored. microorganisms now has 107,126 records (Fungi 45,739); all tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review decisions: microsporidia in Fungi, accepted names, missing protozoa

- all microsporidia in the Fungi (1,394 names of v3.0.1 from P_ to F_, old codes translated by name)
- Enterocytozoon bieneusi and Encephalitozoon cuniculi as accepted names (accepted_name_override)
- Balantidium and Isospora added to MO_RELEVANT_GENERA; Cystoisospora belli as synonym of Isospora belli
- synonym cycles decided: Capillidium and Scolecobasidium are the current genera
- removed the stale data-raw/taxonomy_lpsn0.rds
- microorganisms now has 107,253 records; all tests pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: all items reviewed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rule matching by accepted taxon; Erysipelatoclostridium as synonym of Thomasclavelia

- interpretive_rules(): species named in a rule (literal names, genus_species/fullname) now also match all other
  names of the same accepted taxon, e.g. 'Clostridium ramosum' for Thomasclavelia ramosa; not for genera or regex rules
- Erysipelatoclostridium and its species are synonyms of Thomasclavelia (accepted_name_override); E. merdavium is
  retired by decision of 6 October 2026, and the build removes taxa retired by a human decision
- microorganisms patched with the same logic as the build script (no full rebuild): 107,252 records
- intrinsic_resistant regenerated with the new taxonomy and rule matching: 308,646 rows
- clinical_breakpoints: EUCAST workbook rows merged again, 24 EUCAST 2026 rows moved from C. ramosum to T. ramosa

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy: correct records for names listed twice, no synonyms without a current name, strict data tests

- one record per name keeps the accepted record (e.g. Eggerthella lenta, Gordonia amarae, Anisakis simplex and
  Enterobius vermicularis were synonyms without a current name), links to dropped records are moved
- LPSN lookup reads the taxonomic status and correct name; validly published synonyms are no longer 'accepted'
- synonyms without a current name: linked via LPSN or removed, released codes retired (3,668); records with kept
  children become 'unknown'; the M. tuberculosis complex is protected (decisions of 7 October 2026)
- subspecies codes reserved, current names of synonyms kept when pruning, missing parent records added or moved,
  higher taxonomy harmonised, shared identifiers resolved per domain and rank, pointers and parent links cleaned
- data sets refer only to current names (except intrinsic_resistant); group members only from current names;
  internal CoNS/CoPS/streptococcal lists keep a synonym only with its current name
- intrinsic_resistant, clinical_breakpoints (EUCAST workbooks) and microorganisms.groups regenerated
- 28 new tests in test-data.R; devtools::test(): 1,581 expectations, 0 failures
- open decisions listed in data-raw/taxonomy_review/2026-10/REPORT.md, section 6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy: protect WHO priority genera, follow LPSN for the M. tuberculosis complex, place protist groups in one domain

- all taxa of the genera in MO_WHO_PRIORITY_GENERA are protected, also names that are not validly published
  (replaces the hard-coded M. tuberculosis complex; keeps e.g. Klebsiella quasivariicola, M. canettii, M. orygis)
- LPSN is followed for the M. tuberculosis complex: M. bovis, M. africanum, M. caprae, M. microti and M. pinnipedii
  are heterotypic synonyms of M. tuberculosis (Riojas et al. 2018); a test enforces this
- labyrinthulids in the Chromista (Aplanochytrium), Dictydiaethalium in the Protozoa (genus_domain_move), and the
  rank suffix also between domains ('Acantharia {class}' next to the fungal genus Acantharia)
- devtools::test(): 1,582 expectations, 0 failures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy build no longer depends on the previous build

- intrinsic_resistant is no input of the microorganisms build anymore (it is computed from it); the genera named in
  the interpretive rules are read from the rules instead
- the previous data set no longer makes genera relevant; Debaryomyces added to MO_RELEVANT_GENERA
- new run_after_microorganisms_build.R recreates microorganisms.groups, the EUCAST breakpoints and
  intrinsic_resistant in the right order, and the build runs it at its end
- reproduction_of_intrinsic_resistant.R uses devtools::load_all() instead of the installed package
- effect: 24 never released, not relevant COL records less; devtools::test(): 1,582 expectations, 0 failures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: summary updated to the second round

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MO code retirements: mark the 265 reviewed retirements as reviewed

The Graphium and v2.x Protozoa retirements were reviewed and accepted on 6 October 2026 (see the taxonomy review
report, section 5); 'NEEDS REVIEW' is replaced by 'reviewed by Matthijs S. Berends on 2026-10-06'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:55:12 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 6d54d581b2 (v3.0.1.9100) Fix unit test on wrapped warnings; taxonomy build never touches the version number
- test-data-microorganisms.R: warnings are wrapped to the console width, so the expected patterns now
  allow a line break between words (failed on GitHub Actions since v3.0.1.9097)
- reproduction_of_microorganisms_CLAUDE.md, step 4: the agent never changes the version number or date,
  the human sets them when merging

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 20:32:43 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 1a7f01bfb0 (v3.0.1.9099) Move microorganisms_known_defects.csv out of tests/ to data-raw/microorganisms_files/
- the list of known defects is development data and must not ship with the package
- tests read it from the source repository; mo_repository_root() now walks up from the working
  directory, so the tests also find the repository from an R CMD check directory inside it (GitHub Actions)
- the integrity, known-defects and sentinel tests skip when the source repository is not available
- CLAUDE.md: on local commits to main, the git hooks set the version number and date

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 19:34:07 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 6ab8ff8c33 (v3.0.1.9098) Approve name of species group B_MYCBC_AVIM-C as 'Mycobacterium avium complex'
As curated in data-raw/Species groups.xlsx since July 2023; the released v3.0.1 data still
used 'Mycobacterium avium-intracellulare complex' for the same group of organisms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 14:16:47 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 83a4853936 (v3.0.1.9097) Never remove released taxa; link orthographic variants; registry line in NEWS
- taxonomy build: released taxa (since v2.0.0) that are missing are restored from their
  last release with their old code and last known status, and are exempt from later
  removals (decision by Matthijs S. Berends)
- exceptions are retired on purpose in data-raw/microorganisms_files/mo_code_retirements.csv
  with a reason: other organisms with the same genus name (e.g. the Graphium butterflies,
  wrongly in the Fungi until v3.0.1) and taxa whose genus is now only known in another
  domain; as.mo() translates these codes to NA with that reason
- synonyms without a current name are linked to an accepted name in the same genus that
  only differs by a Latin gender ending (e.g. Candida glabratus -> Nakaseomyces glabratus)
- placeholders such as "(unknown class)" never become records; codes of manually added
  records are matched by domain and name
- integrity rule released_taxon_missing; known defects of the data on main updated

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 14:15:08 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 d683b9e7c6 (v3.0.1.9096) MO code registry: codes never change meaning, are never guessed, and are validated
- data-raw/microorganisms_files/: registry of all MO codes of every release since v2.0.0
  (78,992 codes, built from the git tags), approved renames, and README with the rules;
  decision by Matthijs S. Berends: MO codes from before v2.0.0 are not supported
- as.mo(): codes of earlier releases are translated deterministically via the internal
  MO_RETIRED_CODES (generated from the registry); unknown codes are no longer guessed
  (previously F_TRCHP, Trichophyton in v3.0.1, became Talaromyces chlamydosporus)
- synonym_mo_to_accepted_mo(): a missing target of one source no longer overwrites a valid
  target of another source (e.g. Chaetomium abuense), and no 'renamed to itself'
- taxonomy build: codes are reused and reserved from the registry instead of the development
  data (fixes 40 codes reused in the 2026 development data), also for phylum to family;
  synonym cycles are broken; dangling 'renamed to' identifiers are removed; the build stops
  on any broken integrity rule before saving
- run_microorganisms_build.R: runs the build in chunks at its checkpoints in one R process,
  with progress logging and resuming from any checkpoint; prompt updated accordingly
- tests: shared integrity rules (helper-microorganisms.R) used by build and testthat, registry
  tests over all codes, sentinel organisms, regression tests; current defects of the data on
  main are listed in microorganisms_known_defects.csv until the next rebuild

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 11:45:25 +02:00
Matthijs BerendsandClaude Opus 5.5 dab4c47e86 EUCAST import: blocking rows, note restrictions, PK-PD and parser fixes (#310)
Make the EUCAST Clinical Breakpoint Tables import (2019-2026) fail-safe:

- Add blocking rows (NA breakpoints, reason in `note`) where EUCAST gives
  no breakpoint for an organism (IE, "-", Note, brackets, "except"
  clauses, table scope), so as.sir() no longer falls back to broader taxa
- Apply restrictions from EUCAST notes via a curated rule table with a
  completeness guard (e.g. zone diameters for E. coli only)
- Import PK-PD (non-species related) breakpoints 2019-2023, H.
  parainfluenzae MIC breakpoints, and explicit screening thresholds from
  notes (high-level aminoglycosides, cefoxitin MIC, unidentified CoNS)
- Parser fixes: lost Staphylococcus sheets 2019/2020, note keys and
  bodies, struck-through cells, hidden sheets, missing disk contents,
  organism groups, site vocabulary, MIC dilution levels, off-scale
  breakpoints stored as 0.0001
- Keep EUCAST antifungal breakpoints 2019-2025 from WHONET; code WHONET
  streptomycin as STR1 instead of STR (streptoduocin)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 06:01:58 +02:00
Matthijs Berends 5d9974e2b9 implement EUCAST >= 2019 directly from their source files 2026-10-01 14:53:00 +01:00
dr. M.S. (Matthijs) Berends aebad2ae47 (v3.0.1.9091) unit test 2026-09-08 15:02:07 +02:00
dr. M.S. (Matthijs) Berends 3f50d6b6c3 (v3.0.1.9090) unit test 2026-09-04 19:47:30 +02:00
Matthijs Berends 2c33297cf6 new eucast breakpoints parsing 2026-08-20 13:29:16 +01:00
dr. M.S. (Matthijs) Berends e23d7b4c45 (v3.0.1.9081) fix mo 2026-06-27 15:33:08 +02:00
Matthijs BerendsandClaude 518425311e (v3.0.1.9080) fix(as.mo): resolve abbreviated genus when species has subspecies (#288 follow-up) (#301)
When a genus+species abbreviation like "P. ovale" was used, the previous
bypass (Issue #288) checked sum(sp_exact) == 1, which failed if the species
also had subspecies sharing the epithet (ovale curtisi, ovale wallikeri).
The fix extends the bypass to fire whenever all exact species matches belong
to one genus, collapsing to the species-rank record (subspecies == "") for
genus+species queries and preserving the chosen row for explicit subspecies
queries.

Also extends the data-invariant test to cover all taxonomic rank columns
from domain to subspecies, not just the terminal three.


Claude-Session: https://claude.ai/code/session_01M4fqQYQYJ3drdudkDYNqAY

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-27 15:20:38 +02:00
Matthijs BerendsandClaude Sonnet 4.6 03be4b87fc (v3.0.1.9079) docs: rename remaining 'antibiotic selectors'/'AB selectors' to 'antimicrobial selectors'/'AMR selectors'
* docs: rename remaining 'antibiotic selectors'/'AB selectors' to 'antimicrobial selectors'/'AMR selectors'

Five leftover occurrences of the old terminology updated in the AMR
vignette (section heading + code comment), the amr_selectors test file,
and two roxygen example comments in amr_selectors.R. The auto-generated
man/antimicrobial_selectors.Rd will update on the next devtools::document() run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169LucQ7SHDLHdTnDs1ENhd

* fix

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-27 14:31:58 +02:00
dr. M.S. (Matthijs) Berends c7b17e5833 (v3.0.1.9063) fix old R version 2026-06-24 10:22:56 +02:00
Matthijs Berends 3f9f931777 (v3.0.1.9059) Update taxonomy of microorganisms 2026-06-23 01:38:13 +02:00
dr. M.S. (Matthijs) Berends 0af3f84655 (v3.0.1.9057) website fix 2026-05-02 14:56:26 +02:00
Matthijs BerendsandClaude 24f24ecaf8 Generalise interpretive rules for multi-guideline support (#268) (#283)
* Generalise interpretive rules for multi-guideline support (#268)

- Rename data-raw/eucast_rules.tsv → interpretive_rules.tsv; add rule.provider
  column (value: "EUCAST") to distinguish future CLSI rows
- Rename EUCAST_RULES_DF → INTERPRETIVE_RULES_DF in _pre_commit_checks.R;
  filter by rule.provider == guideline when applying rules in interpretive_rules()
- Rename custom_eucast_rules() → custom_interpretive_rules() with new S3 class
  "custom_interpretive_rules"; old function becomes a deprecated wrapper in
  zz_deprecated.R; backward-compat S3 dispatch shims added for old class
- Remove stop_if(guideline == "CLSI", ...) so clsi_rules() no longer errors
- Add .onLoad shim in zzz.R to create INTERPRETIVE_RULES_DF from EUCAST_RULES_DF
  for transitional compatibility until sysdata.rda is regenerated

https://claude.ai/code/session_01D46BTsfJSPo3HnLWp3PRkP

* Fix namespace load failure: remove assignInNamespace from .onLoad (#268)

assignInNamespace cannot add NEW bindings to a locked package namespace
(R locks namespace bindings before .onLoad runs). Replace the .onLoad
shim with a runtime fallback inside interpretive_rules(): if
INTERPRETIVE_RULES_DF is absent (pre-regeneration sysdata.rda), derive
it from EUCAST_RULES_DF by adding the rule.provider column. This also
fixes the screening_abx line to reuse the already-resolved
interpretive_rules_df_total instead of a bare INTERPRETIVE_RULES_DF
reference.

https://claude.ai/code/session_01D46BTsfJSPo3HnLWp3PRkP

* fixes

* fixes

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-05-01 18:38:51 +01:00
Matthijs BerendsandClaude f7e9294bea Add parallel computing support to antibiogram() and wisca() (#281) (#282)
* Add parallel computing support to antibiogram() and wisca() (#281)

For WISCA: simulations are distributed across (group, chunk) job pairs
via future.apply::future_lapply(), keeping all workers active even when
the regimen count is smaller than nbrOfWorkers(). Sequential fallback
with progress ticker is preserved when parallel = FALSE or workers = 1.

For grouped antibiograms: each group is processed by a separate worker,
mirroring the row-batch approach in as.sir().

Same gate pattern as as.sir() (PR #280): requires a non-sequential
future::plan() to be active; auto-upgrades to parallel = TRUE when a
parallel plan is detected; throws an informative error otherwise.

https://claude.ai/code/session_01FC43syPbzhGmKgrrVNHjnF

* Fix version to 3.0.1.9055 and update CLAUDE.md version formula

Uses origin/${defaultbranch} (with a fetch) instead of the local
branch ref so the commit count is never stale after a merge.

https://claude.ai/code/session_01FC43syPbzhGmKgrrVNHjnF

* Fix non-ASCII characters in antibiogram.R

Replace en/em dashes and non-breaking spaces with ASCII equivalents
to satisfy R CMD check portability requirement.

https://claude.ai/code/session_01FC43syPbzhGmKgrrVNHjnF

* Update auto-generated Rd files after documentation rebuild

https://claude.ai/code/session_01FC43syPbzhGmKgrrVNHjnF

* Move parallel gate to top of antibiogram.default() like sir.R

The gate was inside the wisca==TRUE block, so parallel=TRUE with a
sequential plan was silently ignored for non-WISCA antibiograms.
Now the gate runs unconditionally at the top of the function,
identical to the as.sir() pattern: error on explicit parallel=TRUE
with sequential plan, auto-upgrade when a non-sequential plan is
already active.

https://claude.ai/code/session_01FC43syPbzhGmKgrrVNHjnF

* Fix parallel WISCA returning all NA; strengthen tests; add sequential hint

Bug: lapply() over a factor yields length-1 factor elements (integer
codes), while for() over a factor yields character strings.  The job
list stored j\$group as a factor integer, but the reassembly loop
compared it with identical(j\$group, g) where g was character -- always
FALSE, so no simulation chunks were ever assembled and coverage stayed
NA throughout.

Fix: convert unique_groups to character before building jobs so both
the job list and the reassembly loop use the same type.

Tests: replaced na.rm = TRUE guards with explicit anyNA() checks so the
test suite would have caught the all-NA result immediately.

Also adds a sequential-mode performance hint (analogous to sir.R
lines 1116-1127) when simulations >= 500 and >= 3 regimens.

https://claude.ai/code/session_01FC43syPbzhGmKgrrVNHjnF

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-30 18:41:56 +01:00
Matthijs BerendsandClaude 23beebc6c3 Migrate parallel computing in as.sir() from parallel:: to future/future.apply (#280)
* Migrate parallel computing in as.sir() from parallel:: to future/future.apply

Replace parallel::mclapply() and parallel::parLapply() with
future.apply::future_lapply(), enabling transparent support for any
future backend (multisession, multicore, mirai_multisession, cluster)
on all platforms including Windows.

When parallel = TRUE the function now: (1) respects an active
future::plan() set by the user without overriding it on exit, or
(2) sets a temporary multisession plan with parallelly::availableCores()
and tears it down on exit. The max_cores argument controls worker count
only when no user plan is active.

future and future.apply are added to Suggests in DESCRIPTION.

https://claude.ai/code/session_01M1Jvf2Miu6JL4TQrEh1wS8

* Require user plan() for parallel=TRUE; fix as_wt_nwt false-positive warnings

- parallel = TRUE now errors with a cli-styled message if no non-sequential
  future::plan() is active; users must call e.g. future::plan(future::multisession)
  before using parallel = TRUE (breaking change)
- Removed auto-setup/teardown of multisession plan inside as.sir(), which was
  slow and caused version-mismatch issues with load_all() workflows
- Added as_wt_nwt to the exclusion list in as_sir_method() to suppress
  false-positive "no longer used" warnings during parallel runs
- Fixed pieces_per_col row-batch calculation to use n_workers (total available
  workers from the active plan) instead of n_cores (workers clipped to n_cols),
  so row-batch mode activates correctly when n_cols < n_workers
- Updated @param parallel and @param max_cores roxygen docs; regenerated man/as.sir.Rd
- Updated sequential-mode hint to instruct users to set plan() first

https://claude.ai/code/session_01M1Jvf2Miu6JL4TQrEh1wS8

* fix parallel

* fix parallel

* unit tests

* unit tedts

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-30 08:57:19 +01:00
Matthijs BerendsandClaude 8261b91b24 Fix custom reference_data support in as.sir() (#239) (PR #279)
* Fix custom reference_data support in as.sir() (#239)

- custom guideline names now correctly classify values as R: CLSI convention
  (>= breakpoint_R for MIC, <= for disk) applies only when guideline contains
  "CLSI"; all other guidelines including custom ones use the EUCAST convention
  (> breakpoint_R for MIC, < for disk)
- guideline argument is now optional when reference_data is manually set: if
  omitted or if its value does not match any row in the custom data, all rows
  in reference_data are used; if set to a value present in the data, only
  matching rows are filtered — useful for multi-guideline custom tables
- host = NA in custom reference_data now acts as a host-agnostic fallback
  when no host-specific breakpoint row exists for the current animal species
- updated reference_data argument documentation to explain these conventions

https://claude.ai/code/session_01Q8KtFFGG9qrjAgLJBbxG2U

* Refactor R-classification logic using custom_breakpoints_set flag

Introduce custom_breakpoints_set <- !identical(reference_data, AMR::clinical_breakpoints)
at the top of as_sir_method() and replace all identical() calls inside that
function with this variable.

In the case_when_AMR interpretation blocks (MIC and disk), the R-classification
now has three explicit arms:
- !custom_breakpoints_set & EUCAST guideline -> open interval (> / <)
- !custom_breakpoints_set & CLSI guideline  -> closed interval (>= / <=)
- custom_breakpoints_set                    -> open interval (> / <), always,
  regardless of the guideline name in the custom data (e.g. "CLSI_custom"
  must not accidentally trigger CLSI convention)

https://claude.ai/code/session_01Q8KtFFGG9qrjAgLJBbxG2U

* Fix unit tests for custom reference_data (#239)

- Do not override my_bp$mo / my_bp$ab in tests: assigning plain character
  strips the <mo>/<ab> class, which check_reference_data() rejects. Use the
  mo/ab values already present in the source row instead.
- Use NA_character_ instead of NA for my_bp$host so the host column keeps
  its character class.
- Pass breakpoint_type = "animal" explicitly in the host-fallback test since
  the custom reference_data only contains animal-type breakpoints.

https://claude.ai/code/session_01Q8KtFFGG9qrjAgLJBbxG2U

* Add coerce_reference_data_columns() for lenient reference_data validation

check_reference_data() now returns the (possibly coerced) reference_data and
the call site captures the result so downstream code sees the fixed columns.

A new coerce_reference_data_columns() helper is called before the strict class
check inside check_reference_data(). It coerces columns to the expected types:
- mo  -> as.mo() if not already <mo> class
- ab  -> as.ab() if not already <ab> class
- character columns -> as.character() (e.g. host = NA becomes NA_character_)
- numeric columns  -> as.double()
- logical columns  -> as.logical()

This allows users to build a custom reference_data from a plain data.frame
without having to pre-apply as.mo()/as.ab() or worry about NA column types.

Updated the reference_data roxygen argument to document the auto-coercion and
restored the tests to the simpler form that uses plain character assignments,
relying on the new coercion instead of workarounds.

https://claude.ai/code/session_01Q8KtFFGG9qrjAgLJBbxG2U

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-25 14:38:01 +02:00
Matthijs BerendsandClaude 19157ce718 Fix parallel computing in as.sir.data.frame (#276)
* Fix parallel computing in as.sir.data.frame

Six bugs in parallel = TRUE mode:

1. PSOCK workers (Windows / R < 4.0) never had AMR loaded, so every
   exported/AMR function call failed. Added clusterEvalQ(cl, library(AMR))
   with a graceful fallback to sequential when the package cannot be loaded
   (e.g. dev-only load_all() environments).

2. clusterExport'd AMR_env was a frozen serialised copy; as.sir() on the
   worker wrote to AMR:::AMR_env while run_as_sir_column read from the stale
   copy, so the captured log was always wrong. Fixed by resolving AMR_env
   dynamically via get("AMR_env", envir = asNamespace("AMR")) inside the
   worker function, and removing AMR_env from clusterExport.

3. In the fork-based (mclapply) path each worker inherited the parent's full
   sir_interpretation_history. Capturing the whole log then combining across
   workers duplicated every pre-existing entry. Fixed by recording the log
   row count before the as.sir() call and slicing only the new rows
   afterwards.

4. run_as_sir_column used non-exported internals (%pm>%, pm_pull,
   as.sir.default) that are inaccessible on PSOCK workers after library(AMR).
   Replaced pipe chains with direct as.mic(as.character(x[, col, drop=TRUE]))
   and as.disk(...) calls, and changed as.sir.default() to as.sir() which
   dispatches correctly via S3.

5. With info = TRUE, worker forks printed per-column progress messages
   simultaneously, producing garbled interleaved console output. Per-column
   messages are now suppressed inside workers (effective_info = FALSE) while
   the outer "Running in parallel" / "DONE" messages still appear.

6. Malformed Unicode escape \u00a (3 hex digits) in the "DONE" banner was
   parsed by R as U+00AD (soft hyphen) + "ONE"; corrected to  .

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Add parallel computing tests to test-sir.R

Eight targeted tests verify correctness of the parallel as.sir() path:
identical SIR output vs sequential, matching log row counts, no
pre-existing history duplication, reproducibility across runs, results
consistency across max_cores values, single-column fallback, and no
per-column worker messages leaking when info = TRUE. All pass when only
1 core is available (parallel silently falls back to sequential).

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Fix as.sir() data.frame: preserve already-<sir> columns, exclude metadata

Issue #278: two related bugs in the column-detection / type-assignment pipeline.

Bug 1 – already-<sir> columns deleted on re-run
  Line 886 excluded already-sir columns from the type assignment (they
  stayed type "") causing the result loop to do x[,col] <- NULL, deleting
  them.  Fix: drop the !is.sir() guard so all untyped columns fall through
  to type "sir" and are re-processed correctly.

Bug 2 – metadata columns treated as antibiotics
  as.ab("patient") -> OXY, as.ab("ward") -> PRU.  The column detector
  accepted any column whose name matched an antibiotic code, regardless of
  content.  Fix: for name-matched columns that do not already carry an AMR
  class, also verify content looks like AMR data (all_valid_mics, all-
  numeric, or any SIR-like string).  all_valid_disks() is intentionally
  avoided here because it strips letters from strings (as.disk("Pt_1")==1).

Also adds tools/benchmark_parallel.R: a standalone script that times
sequential vs parallel as.sir() across n=20/200/2000/20000 rows and
saves a ggplot2 PNG to tools/benchmark_parallel.png.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Update benchmark: two-panel script with warm-up and column-count sweep

Previous single-panel benchmark was misleading: the first sequential run
paid one-time cache-warm-up cost (skewing n=20), and only 6 columns were
used so only 6 cores were ever active on a 16-core machine.

New two-panel design:
  Left  – vary rows with 16 fixed AB columns (shows memory-bandwidth
          saturation for large n)
  Right – vary columns with fixed rows (shows the real speedup profile:
          parallel wins when n_cols >> 1)

Also adds a warm-up pass before measurements to eliminate first-call bias.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Optimise parallel as.sir(): row-batch mode when n_cols < n_cores

Previously parallel dispatch only parallelised by column, so a 6-column
dataset on a 16-core machine used at most 6 cores with the other 10 idle.
For large n this also caused memory-bandwidth saturation (each worker did
a full n-row scan of clinical_breakpoints simultaneously).

New row-batch mode (fork path, R >= 4.0, non-Windows):
  pieces_per_col = ceil(n_cores / n_cols)
  Jobs = n_cols × pieces_per_col  (≈ n_cores jobs total)
  Each job: one column × one row slice

Benefits:
  - All cores stay busy regardless of column count
  - Per-worker memory footprint shrinks by pieces_per_col ×
  - Breakpoints lookup cache pressure reduced per worker

PSOCK path (Windows / R < 4.0) is unchanged: per-job serialisation
overhead makes row batching unprofitable there.

run_as_sir_column() gains an optional `rows` parameter (NULL = all rows,
backward-compatible). Results are reassembled via as.sir(c(as.character(.)))
which is safe for already-clean SIR values.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Fix info=FALSE ignored when no breakpoints found in as_sir_method

Operator-precedence bug at line 1601:

  if (isTRUE(info) && nrow(df_unique) < 10 || nrow(breakpoints) == 0)

R evaluates && before ||, so this was equivalent to:

  (isTRUE(info) && nrow(df_unique) < 10) || (nrow(breakpoints) == 0)

When nrow(breakpoints) == 0 (e.g. cefoxitin / flucloxacillin / mupirocin
against E. coli in EUCAST) the intro message was always printed regardless
of info. Fix: add parentheses so info gates both conditions:

  isTRUE(info) && (nrow(df_unique) < 10 || nrow(breakpoints) == 0)

Also pass print = isTRUE(info) to progress_ticker so the progress bar
(which prints intro_txt as its title) is suppressed when info = FALSE.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Fix cli formatting in as.sir() messages

- stop_if for empty ab_cols: wrap as.mic() and as.disk() in
  {.help [{.fun ...}](...)} for clickable links in cli output
- Parallel mode message: use {.field col} formatting for column names
  and quotes = FALSE in vector_and(), consistent with the rest of the
  codebase (avoids double-quoting from both font_bold and quotes="'")

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Use font_bold() inside {.field} for column names in parallel message

Convention: paste0("{.field ", font_bold(col), "}") gives bold green
column names without quotation marks, consistent with the rest of the
codebase (e.g. the 'Cleaning values' message in run_as_sir_column).

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Add collapse = NULL to font_bold() for column name vectors

font_bold() without collapse = NULL joins a vector with "" into a single
string, breaking paste0() element-wise formatting for length > 1 vectors.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

* Add tools/ to .Rbuildignore

Keeps the benchmark script out of the built package tarball.

https://claude.ai/code/session_012DXCXbZUC54Zij1z9bFiHR

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-25 00:34:38 +02:00
dr. M.S. (Matthijs) Berends 225493192c (v3.0.1.9043) fix unit test 2026-04-02 12:24:12 +02:00
dr. M.S. (Matthijs) Berends 3a736bc484 (v3.0.1.9041) add breakpoints 2026 2026-03-30 10:01:49 +02:00