Commit Graph
52 Commits
Author SHA1 Message Date
dr. M.S. (Matthijs) Berends eb9a9c2fa5 (v3.0.1.9149) Fix WHONET codes in microorganisms.codes that referred to unrelated microorganisms, let codes refer to the name they were named after and add a source column, and replace the fuzzy matching of WHONET names by exact matching with a guard against unreviewed remaps
WHONET names are now matched exactly (also outdated names, after removing qualifiers), then on GBIF identifier, and fuzzy results are only kept if consistent with that; otherwise the genus or UNKNOWN is used. A code refers to the record of the name it was named after (e.g. EAE is Enterobacter aerogenes) if that name refers to the same taxon. Every remap of an existing code to another taxon must be listed in data-raw/whonet_remaps_reviewed.csv; the 198 remaps of this rebuild were reviewed.
2026-10-15 21:08:04 +02:00
Matthijs BerendsandClaude Opus 5.5 0ca43df459 Update taxonomy of microorganisms (2026-10) (#312)
* Update taxonomy of microorganisms (2026-10)

Rebuild of the microorganisms data set from COL 2026-04-18 XR, LPSN and BacDive of October 2026
and MycoBank of January 2026, with v3.0.1 as the previous data set: 111,103 records (v3.0.1:
78,679), 0 changed MO codes of existing names, 522 released codes retired (275 to be reviewed).

Script fixes (data-raw/_reproduction_scripts/reproduction_of_microorganisms.R):
- previous data set from the last release; COL read as COL Data Package (NameUsage.tsv)
- keep COL records without authors (a filter on NA dropped them, e.g. Balamuthia mandrillaris)
- genus homonyms: unclassified MycoBank entries are not authoritative; the released domain of a
  genus wins if a source supports it (e.g. Moorella, Serpula, Pirella stay Bacteria)
- protect current names of relevant species instead of whole current genera (data set tripled)
- current names and synonyms only within the same domain and for known names (Brugia -> Lycoperdon)
- MycoBank names in Latin script only; old records outside their source's scope become manually added
- Graphium butterflies of v3.0.1 are not carried over, re-added or restored
- restore released taxa by name and rank; retire older cross-domain homonyms for review
- fill empty families from COL and a commented override list (Ascaris, Balamuthia)
- add Brugia malayi and B. timori; remove Salmonella species synonyms that are serovars
- BacDive: a minority of facultative strain records no longer decides (C. difficile)
- translate codes of other data sets with the development version; review unmatched codes
- scope decisions: COL XR names only if relevant or released; Protozoa follow the Fungi rules

Also: all checkpoints (data-raw/taxonomy*.rds), updated data sets, Bartlett et al. list, accessed
dates in TAXONOMY_VERSION, NEWS, two staphylococci added to CoNS, tests updated for real taxonomic
changes, known defects file removed (all solved), and the findings report with all review files in
data-raw/taxonomy_review/2026-10/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: note on the .dta export and the single commit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix as.mo() for Salmonella serovars written with the species; add S. xeri to CoNS

- "Salmonella enterica (subsp. enterica) serovar Typhi" returned S. bongori, and
  "S. enterica Typhimurium" S. enterica diarizonae; "Paratyphi A/B/C" became serogroups
- Staphylococcus xeri and S. parequorum confirmed as coagulase-negative

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy build: remove lichens (decision of 5 October 2026)

Fungi in the lichen-forming classes and the order Verrucariales are removed unless their genus is
clinically relevant, with the synonyms that point to them (4,155 records); released lichens are
restored. microorganisms now has 107,126 records (Fungi 45,739); all tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review decisions: microsporidia in Fungi, accepted names, missing protozoa

- all microsporidia in the Fungi (1,394 names of v3.0.1 from P_ to F_, old codes translated by name)
- Enterocytozoon bieneusi and Encephalitozoon cuniculi as accepted names (accepted_name_override)
- Balantidium and Isospora added to MO_RELEVANT_GENERA; Cystoisospora belli as synonym of Isospora belli
- synonym cycles decided: Capillidium and Scolecobasidium are the current genera
- removed the stale data-raw/taxonomy_lpsn0.rds
- microorganisms now has 107,253 records; all tests pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: all items reviewed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rule matching by accepted taxon; Erysipelatoclostridium as synonym of Thomasclavelia

- interpretive_rules(): species named in a rule (literal names, genus_species/fullname) now also match all other
  names of the same accepted taxon, e.g. 'Clostridium ramosum' for Thomasclavelia ramosa; not for genera or regex rules
- Erysipelatoclostridium and its species are synonyms of Thomasclavelia (accepted_name_override); E. merdavium is
  retired by decision of 6 October 2026, and the build removes taxa retired by a human decision
- microorganisms patched with the same logic as the build script (no full rebuild): 107,252 records
- intrinsic_resistant regenerated with the new taxonomy and rule matching: 308,646 rows
- clinical_breakpoints: EUCAST workbook rows merged again, 24 EUCAST 2026 rows moved from C. ramosum to T. ramosa

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy: correct records for names listed twice, no synonyms without a current name, strict data tests

- one record per name keeps the accepted record (e.g. Eggerthella lenta, Gordonia amarae, Anisakis simplex and
  Enterobius vermicularis were synonyms without a current name), links to dropped records are moved
- LPSN lookup reads the taxonomic status and correct name; validly published synonyms are no longer 'accepted'
- synonyms without a current name: linked via LPSN or removed, released codes retired (3,668); records with kept
  children become 'unknown'; the M. tuberculosis complex is protected (decisions of 7 October 2026)
- subspecies codes reserved, current names of synonyms kept when pruning, missing parent records added or moved,
  higher taxonomy harmonised, shared identifiers resolved per domain and rank, pointers and parent links cleaned
- data sets refer only to current names (except intrinsic_resistant); group members only from current names;
  internal CoNS/CoPS/streptococcal lists keep a synonym only with its current name
- intrinsic_resistant, clinical_breakpoints (EUCAST workbooks) and microorganisms.groups regenerated
- 28 new tests in test-data.R; devtools::test(): 1,581 expectations, 0 failures
- open decisions listed in data-raw/taxonomy_review/2026-10/REPORT.md, section 6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy: protect WHO priority genera, follow LPSN for the M. tuberculosis complex, place protist groups in one domain

- all taxa of the genera in MO_WHO_PRIORITY_GENERA are protected, also names that are not validly published
  (replaces the hard-coded M. tuberculosis complex; keeps e.g. Klebsiella quasivariicola, M. canettii, M. orygis)
- LPSN is followed for the M. tuberculosis complex: M. bovis, M. africanum, M. caprae, M. microti and M. pinnipedii
  are heterotypic synonyms of M. tuberculosis (Riojas et al. 2018); a test enforces this
- labyrinthulids in the Chromista (Aplanochytrium), Dictydiaethalium in the Protozoa (genus_domain_move), and the
  rank suffix also between domains ('Acantharia {class}' next to the fungal genus Acantharia)
- devtools::test(): 1,582 expectations, 0 failures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy build no longer depends on the previous build

- intrinsic_resistant is no input of the microorganisms build anymore (it is computed from it); the genera named in
  the interpretive rules are read from the rules instead
- the previous data set no longer makes genera relevant; Debaryomyces added to MO_RELEVANT_GENERA
- new run_after_microorganisms_build.R recreates microorganisms.groups, the EUCAST breakpoints and
  intrinsic_resistant in the right order, and the build runs it at its end
- reproduction_of_intrinsic_resistant.R uses devtools::load_all() instead of the installed package
- effect: 24 never released, not relevant COL records less; devtools::test(): 1,582 expectations, 0 failures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Taxonomy review report: summary updated to the second round

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MO code retirements: mark the 265 reviewed retirements as reviewed

The Graphium and v2.x Protozoa retirements were reviewed and accepted on 6 October 2026 (see the taxonomy review
report, section 5); 'NEEDS REVIEW' is replaced by 'reviewed by Matthijs S. Berends on 2026-10-06'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:55:12 +02:00
Matthijs Berends 3f9f931777 (v3.0.1.9059) Update taxonomy of microorganisms 2026-06-23 01:38:13 +02:00
dr. M.S. (Matthijs) Berends 3a736bc484 (v3.0.1.9041) add breakpoints 2026 2026-03-30 10:01:49 +02:00
dr. M.S. (Matthijs) Berends fcd8b95e51 (v2.1.1.9272) mo codes sensititre 2025-05-04 18:50:25 +02:00
dr. M.S. (Matthijs) Berends 9e64d7708e (v2.1.1.9248) unit test fix 2025-04-20 15:27:36 +02:00
dr. M.S. (Matthijs) Berends ea443f7483 (v2.1.1.9247) CLSI/EUCAST 2025! 2025-04-20 12:55:31 +02:00
dr. M.S. (Matthijs) Berends ac1c40d8bb (v2.1.1.9081) HUGE microorganisms update for fungi! 2024-09-29 22:17:56 +02:00
dr. M.S. (Matthijs) Berends 640888f408 (v2.1.1.9064) update all microbial taxonomy, add mycobank, big documentation update 2024-07-16 14:53:17 +02:00
dr. M.S. (Matthijs) Berends 4ffac7e22d (v2.1.1.9051) fix VGS mo code 2024-06-16 11:28:56 +02:00
dr. M.S. (Matthijs) Berends 9bf7584d58 (v2.1.1.9049) new 2024 breakpoints, add AMO, set NI instead of N 2024-06-14 22:39:01 +02:00
dr. M.S. (Matthijs) Berends 3179216c81 (v2.1.1.9047) unit test and index 2024-06-13 07:37:53 +02:00
dr. M.S. (Matthijs) Berends acb534102b new species groups, updated clinical breakpoints 2023-07-08 17:30:05 +02:00
Dr. Matthijs Berends f065945d7b Update clinical breakpoints and fix some as.mo() bugs (#117)
* Updates clinical breakpoints EUCAST/CLSI 2023, fixes #102, fixes #112, fixes #113, fixes #114, fixes #115
* docs
* implement ecoffs
* unit tests
2023-06-22 15:10:59 +02:00
dr. M.S. (Matthijs) Berends c6135d2082 updated microorganism codes 2023-05-26 16:10:01 +02:00
dr. M.S. (Matthijs) Berends ed70f95380 Fix clinical breakpoints 2023-04-14 23:14:34 +02:00
dr. M.S. (Matthijs) Berends 147f9112e9 Fix some WHONET codes 2023-04-14 11:12:26 +02:00
dr. M.S. (Matthijs) Berends 5f3a7694aa microorganisms update, added Salmonella groups 2022-12-16 16:10:43 +01:00
dr. M.S. (Matthijs) Berends 8da2467209 updated taxonomy 2022-12-12 00:14:56 +01:00
dr. M.S. (Matthijs) Berends c2801ba7a1 new rsi_translation 2022-10-22 22:00:15 +02:00
Dr. Matthijs Berends cd2acc4a29 New mo algorithm, prepare for 2.0 2022-10-05 09:12:22 +02:00
dr. M.S. (Matthijs) Berends 71db246d5c unit test fix 2022-08-28 19:17:12 +02:00
dr. M.S. (Matthijs) Berends 303d61b473 new tibble export 2022-08-27 20:49:37 +02:00
dr. M.S. (Matthijs) Berends 37e6e35ec4 (v1.7.1.9051) updated taxonomy, updated git branch name 2021-10-06 13:23:57 +02:00
dr. M.S. (Matthijs) Berends 8f5e5a3fc2 (v1.7.1.9050) fix for as.mo 2021-10-05 14:00:35 +02:00
dr. M.S. (Matthijs) Berends 5679ccdaf9 (v1.6.0.9017) extra system codes 2021-05-04 12:47:33 +02:00
dr. M.S. (Matthijs) Berends ddf88345f1 (v1.5.0.9028) Updated taxonomy until March 2021 2021-03-04 23:28:32 +01:00
dr. M.S. (Matthijs) Berends d014955ce0 (v.1.5.0.9000) implementation of EUCAST rules v11 (2021) 2021-01-12 22:08:04 +01:00
dr. M.S. (Matthijs) Berends 5e45fdcf2a (v1.3.0.9007) tibble printing 2020-08-26 11:33:54 +02:00
dr. M.S. (Matthijs) Berends 86d44054f0 (v1.1.0.9020) updated taxonomy 2020-05-27 16:37:49 +02:00
dr. M.S. (Matthijs) Berends 19703eb5d3 (v1.1.0.9012) lose dependencies 2020-05-19 12:08:49 +02:00
dr. M.S. (Matthijs) Berends d1cb7d3b6f (v1.0.1.9005) as.mo() improvements 2020-04-13 21:09:56 +02:00
dr. M.S. (Matthijs) Berends c7a76ba713 (v0.9.0.9020) as.mo() improvement 2020-02-09 22:04:29 +01:00
dr. M.S. (Matthijs) Berends 9a3891efe4 (v0.9.0.9014) Machine reading guidelines 2020-01-27 11:05:39 +01:00
dr. M.S. (Matthijs) Berends 7c28b392b1 (v0.8.0.9035) add more mo codes 2019-11-24 22:48:56 +01:00
dr. M.S. (Matthijs) Berends 3e5ab53209 (v0.8.0.9002) eucast_rules() fix for S. maltophilia 2019-10-23 14:48:25 +02:00
dr. M.S. (Matthijs) Berends 8ce252fbc5 (v0.7.1.9083) LIS code improvement 2019-09-23 17:32:05 +02:00
dr. M.S. (Matthijs) Berends 66d405ff57 (v0.7.1.9080) new LIS codes 2019-09-22 17:19:59 +02:00
dr. M.S. (Matthijs) Berends 3596adb295 (v0.7.1.9076) mo codes 2019-09-20 12:33:05 +02:00
dr. M.S. (Matthijs) Berends e2aa4f996b (v0.7.1.9075) new microorganism codes 2019-09-18 15:46:09 +02:00
dr. M.S. (Matthijs) Berends 3f5828e74a (v0.7.1.9034) mo.codes fix for Dracunculus spp. 2019-08-09 23:45:26 +02:00
dr. M.S. (Matthijs) Berends 9ac5a34b36 (v0.7.1.9014) MO codes update 2019-07-16 16:14:11 +02:00
dr. M.S. (Matthijs) Berends 254745061c (v0.7.0.9008) T. vaginalis, rsi_df 2019-06-13 14:28:46 +02:00
dr. M.S. (Matthijs) Berends 27380fa021 fixes for microorganisms.codes 2019-05-28 16:50:40 +02:00
dr. M.S. (Matthijs) Berends 4c3cf85a65 ab_info, other bug fixes 2019-05-16 21:20:00 +02:00
dr. M.S. (Matthijs) Berends 68cc7ef0d0 new antibiotics 2019-05-10 16:44:59 +02:00
dr. M.S. (Matthijs) Berends c6a12266e7 DSMZ data 2019-03-18 14:29:41 +01:00
dr. M.S. (Matthijs) Berends 504a042fba uncertainty levels, new WHONET codes 2019-03-12 12:19:27 +01:00
dr. M.S. (Matthijs) Berends 54f8258f8c Catalogue of Life, replaces ITIS 2019-02-18 02:33:37 +01:00
dr. M.S. (Matthijs) Berends efb2af9e3a benchmarks plots 2019-02-14 10:23:51 +01:00