Commit Graph
45 Commits
Author SHA1 Message Date
Matthijs BerendsandClaude Opus 5.5 0ca43df459 Update taxonomy of microorganisms (2026-10) (#312)
* Update taxonomy of microorganisms (2026-10)

Rebuild of the microorganisms data set from COL 2026-04-18 XR, LPSN and BacDive of October 2026
and MycoBank of January 2026, with v3.0.1 as the previous data set: 111,103 records (v3.0.1:
78,679), 0 changed MO codes of existing names, 522 released codes retired (275 to be reviewed).

Script fixes (data-raw/_reproduction_scripts/reproduction_of_microorganisms.R):
- previous data set from the last release; COL read as COL Data Package (NameUsage.tsv)
- keep COL records without authors (a filter on NA dropped them, e.g. Balamuthia mandrillaris)
- genus homonyms: unclassified MycoBank entries are not authoritative; the released domain of a
  genus wins if a source supports it (e.g. Moorella, Serpula, Pirella stay Bacteria)
- protect current names of relevant species instead of whole current genera (data set tripled)
- current names and synonyms only within the same domain and for known names (Brugia -> Lycoperdon)
- MycoBank names in Latin script only; old records outside their source's scope become manually added
- Graphium butterflies of v3.0.1 are not carried over, re-added or restored
- restore released taxa by name and rank; retire older cross-domain homonyms for review
- fill empty families from COL and a commented override list (Ascaris, Balamuthia)
- add Brugia malayi and B. timori; remove Salmonella species synonyms that are serovars
- BacDive: a minority of facultative strain records no longer decides (C. difficile)
- translate codes of other data sets with the development version; review unmatched codes
- scope decisions: COL XR names only if relevant or released; Protozoa follow the Fungi rules

Also: all checkpoints (data-raw/taxonomy*.rds), updated data sets, Bartlett et al. list, accessed
dates in TAXONOMY_VERSION, NEWS, two staphylococci added to CoNS, tests updated for real taxonomic
changes, known defects file removed (all solved), and the findings report with all review files in
data-raw/taxonomy_review/2026-10/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: note on the .dta export and the single commit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix as.mo() for Salmonella serovars written with the species; add S. xeri to CoNS

- "Salmonella enterica (subsp. enterica) serovar Typhi" returned S. bongori, and
  "S. enterica Typhimurium" S. enterica diarizonae; "Paratyphi A/B/C" became serogroups
- Staphylococcus xeri and S. parequorum confirmed as coagulase-negative

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy build: remove lichens (decision of 5 October 2026)

Fungi in the lichen-forming classes and the order Verrucariales are removed unless their genus is
clinically relevant, with the synonyms that point to them (4,155 records); released lichens are
restored. microorganisms now has 107,126 records (Fungi 45,739); all tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review decisions: microsporidia in Fungi, accepted names, missing protozoa

- all microsporidia in the Fungi (1,394 names of v3.0.1 from P_ to F_, old codes translated by name)
- Enterocytozoon bieneusi and Encephalitozoon cuniculi as accepted names (accepted_name_override)
- Balantidium and Isospora added to MO_RELEVANT_GENERA; Cystoisospora belli as synonym of Isospora belli
- synonym cycles decided: Capillidium and Scolecobasidium are the current genera
- removed the stale data-raw/taxonomy_lpsn0.rds
- microorganisms now has 107,253 records; all tests pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: all items reviewed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rule matching by accepted taxon; Erysipelatoclostridium as synonym of Thomasclavelia

- interpretive_rules(): species named in a rule (literal names, genus_species/fullname) now also match all other
  names of the same accepted taxon, e.g. 'Clostridium ramosum' for Thomasclavelia ramosa; not for genera or regex rules
- Erysipelatoclostridium and its species are synonyms of Thomasclavelia (accepted_name_override); E. merdavium is
  retired by decision of 6 October 2026, and the build removes taxa retired by a human decision
- microorganisms patched with the same logic as the build script (no full rebuild): 107,252 records
- intrinsic_resistant regenerated with the new taxonomy and rule matching: 308,646 rows
- clinical_breakpoints: EUCAST workbook rows merged again, 24 EUCAST 2026 rows moved from C. ramosum to T. ramosa

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy: correct records for names listed twice, no synonyms without a current name, strict data tests

- one record per name keeps the accepted record (e.g. Eggerthella lenta, Gordonia amarae, Anisakis simplex and
  Enterobius vermicularis were synonyms without a current name), links to dropped records are moved
- LPSN lookup reads the taxonomic status and correct name; validly published synonyms are no longer 'accepted'
- synonyms without a current name: linked via LPSN or removed, released codes retired (3,668); records with kept
  children become 'unknown'; the M. tuberculosis complex is protected (decisions of 7 October 2026)
- subspecies codes reserved, current names of synonyms kept when pruning, missing parent records added or moved,
  higher taxonomy harmonised, shared identifiers resolved per domain and rank, pointers and parent links cleaned
- data sets refer only to current names (except intrinsic_resistant); group members only from current names;
  internal CoNS/CoPS/streptococcal lists keep a synonym only with its current name
- intrinsic_resistant, clinical_breakpoints (EUCAST workbooks) and microorganisms.groups regenerated
- 28 new tests in test-data.R; devtools::test(): 1,581 expectations, 0 failures
- open decisions listed in data-raw/taxonomy_review/2026-10/REPORT.md, section 6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy: protect WHO priority genera, follow LPSN for the M. tuberculosis complex, place protist groups in one domain

- all taxa of the genera in MO_WHO_PRIORITY_GENERA are protected, also names that are not validly published
  (replaces the hard-coded M. tuberculosis complex; keeps e.g. Klebsiella quasivariicola, M. canettii, M. orygis)
- LPSN is followed for the M. tuberculosis complex: M. bovis, M. africanum, M. caprae, M. microti and M. pinnipedii
  are heterotypic synonyms of M. tuberculosis (Riojas et al. 2018); a test enforces this
- labyrinthulids in the Chromista (Aplanochytrium), Dictydiaethalium in the Protozoa (genus_domain_move), and the
  rank suffix also between domains ('Acantharia {class}' next to the fungal genus Acantharia)
- devtools::test(): 1,582 expectations, 0 failures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy build no longer depends on the previous build

- intrinsic_resistant is no input of the microorganisms build anymore (it is computed from it); the genera named in
  the interpretive rules are read from the rules instead
- the previous data set no longer makes genera relevant; Debaryomyces added to MO_RELEVANT_GENERA
- new run_after_microorganisms_build.R recreates microorganisms.groups, the EUCAST breakpoints and
  intrinsic_resistant in the right order, and the build runs it at its end
- reproduction_of_intrinsic_resistant.R uses devtools::load_all() instead of the installed package
- effect: 24 never released, not relevant COL records less; devtools::test(): 1,582 expectations, 0 failures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: summary updated to the second round

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MO code retirements: mark the 265 reviewed retirements as reviewed

The Graphium and v2.x Protozoa retirements were reviewed and accepted on 6 October 2026 (see the taxonomy review
report, section 5); 'NEEDS REVIEW' is replaced by 'reviewed by Matthijs S. Berends on 2026-10-06'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:55:12 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 4aafe9a676 (v3.0.1.9103) Replace dead EC Union Register URLs; URL check no longer includes closing braces
The pre-commit URL extractor captured the closing brace of \url{}, which
made the EC server redirect to the dead health.ec.europa.eu domain, after
which urlchecker::url_update() wrote that dead URL back into R/ab.R and R/av.R.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 19:53:40 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 83a4853936 (v3.0.1.9097) Never remove released taxa; link orthographic variants; registry line in NEWS
- taxonomy build: released taxa (since v2.0.0) that are missing are restored from their
  last release with their old code and last known status, and are exempt from later
  removals (decision by Matthijs S. Berends)
- exceptions are retired on purpose in data-raw/microorganisms_files/mo_code_retirements.csv
  with a reason: other organisms with the same genus name (e.g. the Graphium butterflies,
  wrongly in the Fungi until v3.0.1) and taxa whose genus is now only known in another
  domain; as.mo() translates these codes to NA with that reason
- synonyms without a current name are linked to an accepted name in the same genus that
  only differs by a Latin gender ending (e.g. Candida glabratus -> Nakaseomyces glabratus)
- placeholders such as "(unknown class)" never become records; codes of manually added
  records are matched by domain and name
- integrity rule released_taxon_missing; known defects of the data on main updated

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 14:15:08 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 d683b9e7c6 (v3.0.1.9096) MO code registry: codes never change meaning, are never guessed, and are validated
- data-raw/microorganisms_files/: registry of all MO codes of every release since v2.0.0
  (78,992 codes, built from the git tags), approved renames, and README with the rules;
  decision by Matthijs S. Berends: MO codes from before v2.0.0 are not supported
- as.mo(): codes of earlier releases are translated deterministically via the internal
  MO_RETIRED_CODES (generated from the registry); unknown codes are no longer guessed
  (previously F_TRCHP, Trichophyton in v3.0.1, became Talaromyces chlamydosporus)
- synonym_mo_to_accepted_mo(): a missing target of one source no longer overwrites a valid
  target of another source (e.g. Chaetomium abuense), and no 'renamed to itself'
- taxonomy build: codes are reused and reserved from the registry instead of the development
  data (fixes 40 codes reused in the 2026 development data), also for phylum to family;
  synonym cycles are broken; dangling 'renamed to' identifiers are removed; the build stops
  on any broken integrity rule before saving
- run_microorganisms_build.R: runs the build in chunks at its checkpoints in one R process,
  with progress logging and resuming from any checkpoint; prompt updated accordingly
- tests: shared integrity rules (helper-microorganisms.R) used by build and testthat, registry
  tests over all codes, sentinel organisms, regression tests; current defects of the data on
  main are listed in microorganisms_known_defects.csv until the next rebuild

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 11:45:25 +02:00
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 ca5fa333ca (v3.0.1.9095) Taxonomy build: fix root causes of missing and misclassified taxa, add unattended run prompt
Review of reproduction_of_microorganisms.R against the 2026 intermediates:
- genus homonyms: overrides, LPSN/MycoBank first, status within LPSN/MycoBank,
  move misplaced protists, remove true homonyms (Necator, Capillaria, Graphium)
- keep clinically relevant names that are not validly published (Tropheryma)
- one set of relevant genera for all filters; 30 genera added to MO_RELEVANT_GENERA
- MycoBank classification parsed from all columns; empty values never overwrite
- GBIF synonyms in another genus get the lineage of their own genus
- prevalence no longer copied from genus; current names inherit from synonyms
- inferred records follow the relevance filters (no more unrelated lichens)
- ASCII names before deduplication (12 duplicate full names in 2026)
- MO codes: robust code parsing, reserved old codes, single allocator,
  domain column used directly (2026 rename would now fail)
- name clean-up applied to taxonomy before MO codes, BacDive rows filled
- unattended running: review() instead of View(), CSV export of reviews,
  LPSN cache and retries, stops when raw source files are missing
- reproduction_of_microorganisms_CLAUDE.md: prompt for an unattended run with
  findings report and checklist for human review in the PR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 09:06:27 +02:00
Matthijs BerendsandClaude Opus 5.5 dab4c47e86 EUCAST import: blocking rows, note restrictions, PK-PD and parser fixes (#310)
Make the EUCAST Clinical Breakpoint Tables import (2019-2026) fail-safe:

- Add blocking rows (NA breakpoints, reason in `note`) where EUCAST gives
  no breakpoint for an organism (IE, "-", Note, brackets, "except"
  clauses, table scope), so as.sir() no longer falls back to broader taxa
- Apply restrictions from EUCAST notes via a curated rule table with a
  completeness guard (e.g. zone diameters for E. coli only)
- Import PK-PD (non-species related) breakpoints 2019-2023, H.
  parainfluenzae MIC breakpoints, and explicit screening thresholds from
  notes (high-level aminoglycosides, cefoxitin MIC, unidentified CoNS)
- Parser fixes: lost Staphylococcus sheets 2019/2020, note keys and
  bodies, struck-through cells, hidden sheets, missing disk contents,
  organism groups, site vocabulary, MIC dilution levels, off-scale
  breakpoints stored as 0.0001
- Keep EUCAST antifungal breakpoints 2019-2025 from WHONET; code WHONET
  streptomycin as STR1 instead of STR (streptoduocin)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 06:01:58 +02:00
Matthijs Berends 2c33297cf6 new eucast breakpoints parsing 2026-08-20 13:29:16 +01:00
Matthijs Berends 3f9f931777 (v3.0.1.9059) Update taxonomy of microorganisms 2026-06-23 01:38:13 +02:00
Matthijs BerendsandClaude 24f24ecaf8 Generalise interpretive rules for multi-guideline support (#268) (#283)
* Generalise interpretive rules for multi-guideline support (#268)

- Rename data-raw/eucast_rules.tsv → interpretive_rules.tsv; add rule.provider
  column (value: "EUCAST") to distinguish future CLSI rows
- Rename EUCAST_RULES_DF → INTERPRETIVE_RULES_DF in _pre_commit_checks.R;
  filter by rule.provider == guideline when applying rules in interpretive_rules()
- Rename custom_eucast_rules() → custom_interpretive_rules() with new S3 class
  "custom_interpretive_rules"; old function becomes a deprecated wrapper in
  zz_deprecated.R; backward-compat S3 dispatch shims added for old class
- Remove stop_if(guideline == "CLSI", ...) so clsi_rules() no longer errors
- Add .onLoad shim in zzz.R to create INTERPRETIVE_RULES_DF from EUCAST_RULES_DF
  for transitional compatibility until sysdata.rda is regenerated

https://claude.ai/code/session_01D46BTsfJSPo3HnLWp3PRkP

* Fix namespace load failure: remove assignInNamespace from .onLoad (#268)

assignInNamespace cannot add NEW bindings to a locked package namespace
(R locks namespace bindings before .onLoad runs). Replace the .onLoad
shim with a runtime fallback inside interpretive_rules(): if
INTERPRETIVE_RULES_DF is absent (pre-regeneration sysdata.rda), derive
it from EUCAST_RULES_DF by adding the rule.provider column. This also
fixes the screening_abx line to reuse the already-resolved
interpretive_rules_df_total instead of a bare INTERPRETIVE_RULES_DF
reference.

https://claude.ai/code/session_01D46BTsfJSPo3HnLWp3PRkP

* fixes

* fixes

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-05-01 18:38:51 +01:00
Matthijs Berends 8ff5d4472a Add add_if_missing parameter to control NA handling in interpretive rules (#264) 2026-04-21 21:53:43 +02:00
dr. M.S. (Matthijs) Berends 353eaa3f38 (v3.0.1.9033) add ionophores(), clorobiocin, aminocoumarins group 2026-03-09 11:57:36 +01:00
dr. M.S. (Matthijs) Berends b6211931f8 (v3.0.1.9022) add ceftibuten/avibactam (CTA), kasugamycin (KAS), ostreogrycin (OST), thiostrepton (THS), xeruborbactam (XER), zorbamycin (ZOR) 2026-03-03 15:41:08 +01:00
dr. M.S. (Matthijs) Berends 2df2911cf4 (v3.0.1.9018) fixes #249
updates AB groups
2026-01-16 10:57:03 +01:00
dr. M.S. (Matthijs) Berends cfbbfb4fa5 (v3.0.1.9007) fix #246 2026-01-06 23:08:50 +01:00
dr. M.S. (Matthijs) Berends 72db2b2562 (v3.0.0.9003) eucast_rules fix, new tidymodels integration 2025-06-13 14:03:21 +02:00
dr. M.S. (Matthijs) Berends 5667ce3eae (v2.1.1.9287) fix antibiotics 2025-06-01 12:17:47 +02:00
dr. M.S. (Matthijs) Berends dc5559a2c4 (v2.1.1.9270) Add support for Korean, fix ATCs 2025-05-04 14:24:43 +02:00
dr. M.S. (Matthijs) Berends b8f0f64287 (v2.1.1.9269) Support 7 new languages 2025-05-02 19:25:19 +02:00
dr. M.S. (Matthijs) Berends 30caa5e654 (v2.1.1.9262) fix knit printing 2025-04-29 16:31:38 +02:00
dr. M.S. (Matthijs) Berends 6135805455 (v2.1.1.9251) fix colour on GitHub docs 2025-04-21 16:04:01 +02:00
dr. M.S. (Matthijs) Berends 4a336d040c (v2.1.1.9250) Automated README and index.md 2025-04-21 15:37:26 +02:00
dr. M.S. (Matthijs) Berends 40d7a971c3 (v2.1.1.9236) documentation 2025-04-12 11:46:42 +02:00
dr. M.S. (Matthijs) Berends 36fd99e1f4 (v2.1.1.9235) New website! 2025-04-08 15:54:30 +01:00
dr. M.S. (Matthijs) Berends 49da312506 (v2.1.1.9228) repo cleaning 2025-03-28 11:17:49 +01:00
dr. M.S. (Matthijs) Berends 969a42cc8c (v2.1.1.9226) update intrinsic_resistant, unit test fixes 2025-03-27 15:20:22 +01:00
dr. M.S. (Matthijs) Berends e6f88241b2 (v2.1.1.9225) fix geom_hline()/_vline() in MIC plotting, add EUCAT 1.2 in full, add London contribs, fix mo codes, add Kleb pneu complex 2025-03-26 17:19:17 +01:00
dr. M.S. (Matthijs) Berends 755e6f43b7 (v2.1.1.9219) unit tests 2025-03-19 16:06:23 +01:00
dr. M.S. (Matthijs) Berends 7631e82ae6 (v2.1.1.9206) update precommit hook 2025-03-15 16:17:27 +01:00
dr. M.S. (Matthijs) Berends ede1cdfd99 (v2.1.1.9205) Fix python wrapper 2025-03-15 16:14:29 +01:00
dr. M.S. (Matthijs) Berends e134e01418 (v2.1.1.9198) support eucast expert rules 14 and 13.1 2025-03-14 10:10:35 +01:00
dr. M.S. (Matthijs) Berends 067a8aca66 (v2.1.1.9192) fix fluoroquinolones and add bleomycin 2025-03-10 17:04:35 +01:00
dr. M.S. (Matthijs) Berends a2c2be23c1 (v2.1.1.9190) antibiotics deprecation in antibiogram() 2025-03-09 10:41:11 +01:00
dr. M.S. (Matthijs) Berends f7938289eb (v2.1.1.9186) replace antibiotics with antimicrobials! 2025-03-07 20:43:26 +01:00
dr. M.S. (Matthijs) Berends 07efc292bc (v2.1.1.9163) cleanup 2025-02-27 14:04:29 +01:00
dr. M.S. (Matthijs) Berends 8ba2e4ed94 (v2.1.1.9141) new AMR selectors, eucast overwrite arg 2025-02-07 18:01:22 +01:00
dr. M.S. (Matthijs) Berends 08ddbaa930 (v2.1.1.9123) add EFF code to antibiotics data set 2025-01-15 16:14:09 +01:00
dr. M.S. (Matthijs) Berends 0488d00f20 (v2.1.1.9111) add betalactams_with_inhibitor(), fixes #175 2024-12-06 15:44:20 +01:00
dr. M.S. (Matthijs) Berends 9fb891eee2 (v2.1.1.9087) update unit tests 2024-10-04 15:28:44 +02:00
dr. M.S. (Matthijs) Berends 50a9f8f0e0 (v2.1.1.9086) website update 2024-10-02 10:20:05 +02:00
dr. M.S. (Matthijs) Berends 681fe011fe (v2.1.1.9082) algorithm updates 2024-09-30 18:46:55 +02:00
dr. M.S. (Matthijs) Berends ac1c40d8bb (v2.1.1.9081) HUGE microorganisms update for fungi! 2024-09-29 22:17:56 +02:00
dr. M.S. (Matthijs) Berends 7f344836ea (v2.1.1.9064) unit tests 2024-07-16 15:55:58 +02:00
dr. M.S. (Matthijs) Berends 640888f408 (v2.1.1.9064) update all microbial taxonomy, add mycobank, big documentation update 2024-07-16 14:53:17 +02:00
dr. M.S. (Matthijs) Berends af74e1d4f2 (v2.1.1.9041) antibiotics update 2024-06-08 17:35:25 +02:00
dr. M.S. (Matthijs) Berends 0d8a91db49 rename pre-commit hook to pre-commit checks (no-check) 2024-04-23 10:55:48 +02:00