mirror of
https://github.com/msberends/AMR.git
synced 2026-11-16 04:20:53 +01:00
(v3.0.1.9096) MO code registry: codes never change meaning, are never guessed, and are validated
- data-raw/microorganisms_files/: registry of all MO codes of every release since v2.0.0 (78,992 codes, built from the git tags), approved renames, and README with the rules; decision by Matthijs S. Berends: MO codes from before v2.0.0 are not supported - as.mo(): codes of earlier releases are translated deterministically via the internal MO_RETIRED_CODES (generated from the registry); unknown codes are no longer guessed (previously F_TRCHP, Trichophyton in v3.0.1, became Talaromyces chlamydosporus) - synonym_mo_to_accepted_mo(): a missing target of one source no longer overwrites a valid target of another source (e.g. Chaetomium abuense), and no 'renamed to itself' - taxonomy build: codes are reused and reserved from the registry instead of the development data (fixes 40 codes reused in the 2026 development data), also for phylum to family; synonym cycles are broken; dangling 'renamed to' identifiers are removed; the build stops on any broken integrity rule before saving - run_microorganisms_build.R: runs the build in chunks at its checkpoints in one R process, with progress logging and resuming from any checkpoint; prompt updated accordingly - tests: shared integrity rules (helper-microorganisms.R) used by build and testthat, registry tests over all codes, sentinel organisms, regression tests; current defects of the data on main are listed in microorganisms_known_defects.csv until the next rebuild Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
ca5fa333ca
commit
d683b9e7c6
16 files changed
+80516
-246
No files matched your search
@@ -0,0 +1,39 @@
|
||||
# MO code registry
|
||||
|
||||
MO codes (such as `B_ESCHR_COLI`) are stored by users in their own data, sometimes for years. A code must
|
||||
therefore **always mean the same taxon**, in every future version of the AMR package. This folder contains
|
||||
the registry that guarantees this.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `mo_code_registry.csv` | Every MO code that was ever part of a **released** version of the AMR package since v2.0.0, with the taxon it denotes (name, rank, domain, genus, species, subspecies), the first and last release it was in, and older names of the same code (e.g. a corrected spelling). |
|
||||
| `build_mo_code_registry.R` | Recreates `mo_code_registry.csv` from the git tags of all releases. Run it after every new release. |
|
||||
| `mo_code_renames.csv` | The only allowed exceptions: registered codes that get another name, because it is the same taxon (e.g. a corrected spelling or a renamed species group). Every row needs `approved_by` and `approved_date`. |
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Only releases count.** Development versions (the main branch) are never part of the registry. Codes that
|
||||
only existed in a development version carry no weight and may be reassigned.
|
||||
2. **Codes from before v2.0.0 are not supported.** Decision by Matthijs S. Berends, 3 October 2026: MO codes
|
||||
from releases before v2.0.0 (12 March 2023) are not supported. Every code in all releases since v2.0.0 has
|
||||
always meant the same taxon (checked on 3 October 2026), so the registry starts without conflicts.
|
||||
3. **A registered code is never given to another taxon.** The taxonomy build
|
||||
(`data-raw/_reproduction_scripts/reproduction_of_microorganisms.R`) reuses the code of every taxon in the
|
||||
registry (most recent release first), never gives a registered code to a new taxon, and stops if a
|
||||
registered code would get another name than in the registry, unless that is listed and approved in
|
||||
`mo_code_renames.csv`.
|
||||
4. **A registered code that is no longer in the data still works.** `as.mo()` translates it to the current
|
||||
code of the same taxon (by its name, so also if the taxon moved to another domain), using an internal lookup
|
||||
table (`MO_RETIRED_CODES`) that is created from the registry in `data-raw/_pre_commit_checks.R`. If the
|
||||
taxon is no longer in the data at all, the result is `NA` with a warning. The package never guesses.
|
||||
5. **Tests enforce all of this.** `tests/testthat/test-data-microorganisms.R` checks the current data against
|
||||
the registry and the renames, and checks that every registered code resolves to its own taxon.
|
||||
|
||||
## After a release
|
||||
|
||||
1. Make sure the git tag of the release exists (e.g. `v3.1.0`).
|
||||
2. Run `source("data-raw/microorganisms_files/build_mo_code_registry.R")` from the root of the repository.
|
||||
3. Review the changes to `mo_code_registry.csv` (`git diff`): only new codes may be added, and existing codes
|
||||
may only get a later `last_release`. Commit the result.
|
||||
@@ -0,0 +1,113 @@
|
||||
# ==================================================================== #
|
||||
# TITLE: #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data #
|
||||
# #
|
||||
# SOURCE CODE: #
|
||||
# https://github.com/msberends/AMR #
|
||||
# #
|
||||
# PLEASE CITE THIS SOFTWARE AS: #
|
||||
# Berends MS, Luz CF, Friedrich AW, et al. (2022). #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data. #
|
||||
# Journal of Statistical Software, 104(3), 1-31. #
|
||||
# https://doi.org/10.18637/jss.v104.i03 #
|
||||
# #
|
||||
# Developed at the University of Groningen and the University Medical #
|
||||
# Center Groningen in the Netherlands, in collaboration with many #
|
||||
# colleagues from around the world, see our website. #
|
||||
# #
|
||||
# This R package is free software; you can freely use and distribute #
|
||||
# it for both personal and commercial purposes under the terms of the #
|
||||
# GNU General Public License version 2.0 (GNU GPL-2), as published by #
|
||||
# the Free Software Foundation. #
|
||||
# We created this package for both routine data analysis and academic #
|
||||
# research and it was publicly released in the hope that it will be #
|
||||
# useful, but it comes WITHOUT ANY WARRANTY OR LIABILITY. #
|
||||
# #
|
||||
# Visit our website for the full manual and a complete tutorial about #
|
||||
# how to conduct AMR data analysis: https://amr-for-r.org #
|
||||
# ==================================================================== #
|
||||
|
||||
# This script (re)creates data-raw/microorganisms_files/mo_code_registry.csv: the registry of all MO codes
|
||||
# that were ever part of a released version of the AMR package, since v2.0.0. See the README.md in this
|
||||
# folder for the rules.
|
||||
#
|
||||
# Run it from the root of the repository, after every new release (once its git tag exists):
|
||||
# source("data-raw/microorganisms_files/build_mo_code_registry.R")
|
||||
# It always rebuilds the whole registry from the git tags, so it is reproducible and can be run at any time.
|
||||
# Development versions (the main branch) are never part of the registry.
|
||||
|
||||
library(dplyr)
|
||||
devtools::load_all(".", quiet = TRUE) # for %like_case%
|
||||
|
||||
registry_file <- "data-raw/microorganisms_files/mo_code_registry.csv"
|
||||
|
||||
# Decision by Matthijs S. Berends, 3 October 2026: MO codes from releases before v2.0.0 (12 March 2023)
|
||||
# are not supported.
|
||||
first_supported_release <- "2.0.0"
|
||||
|
||||
# all release tags (vX.Y.Z, so no development versions) since the first supported release
|
||||
tags <- system2("git", "tag", stdout = TRUE)
|
||||
tags <- tags[tags %like_case% "^v[0-9]+[.][0-9]+[.][0-9]+$"]
|
||||
tags <- tags[numeric_version(sub("^v", "", tags)) >= numeric_version(first_supported_release)]
|
||||
tags <- tags[order(numeric_version(sub("^v", "", tags)))]
|
||||
message("Releases: ", toString(tags))
|
||||
|
||||
read_release <- function(tag) {
|
||||
tmp <- tempfile(fileext = ".rda")
|
||||
on.exit(unlink(tmp))
|
||||
status <- system2("git", c("show", paste0(tag, ":data/microorganisms.rda")), stdout = tmp)
|
||||
if (!identical(status, 0L)) {
|
||||
stop("Could not read data/microorganisms.rda of ", tag, call. = FALSE)
|
||||
}
|
||||
env <- new.env()
|
||||
load(tmp, envir = env)
|
||||
mo <- as.data.frame(env$microorganisms, stringsAsFactors = FALSE)
|
||||
# until v3.0.1, `kingdom` contained what is now called `domain`
|
||||
if (!"domain" %in% colnames(mo)) {
|
||||
mo$domain <- mo$kingdom
|
||||
mo$domain[mo$domain == "(unknown kingdom)"] <- "(unknown domain)"
|
||||
}
|
||||
data.frame(
|
||||
release = tag,
|
||||
mo = as.character(mo$mo),
|
||||
fullname = mo$fullname,
|
||||
rank = mo$rank,
|
||||
domain = mo$domain,
|
||||
genus = mo$genus,
|
||||
species = mo$species,
|
||||
subspecies = mo$subspecies,
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
}
|
||||
|
||||
all_releases <- bind_rows(lapply(tags, read_release)) %>%
|
||||
mutate(release_order = match(release, tags))
|
||||
|
||||
# one row per MO code; the name, rank and taxonomy are taken from the most recent release that contained the code
|
||||
registry <- all_releases %>%
|
||||
group_by(mo) %>%
|
||||
arrange(desc(release_order), .by_group = TRUE) %>%
|
||||
summarise(
|
||||
# other names the code had in older releases (e.g. a corrected subspecies name), for transparency
|
||||
# (must come first, since `fullname` is overwritten below)
|
||||
previous_names = paste(setdiff(unique(fullname), first(fullname)), collapse = "; "),
|
||||
fullname = first(fullname),
|
||||
rank = first(rank),
|
||||
domain = first(domain),
|
||||
genus = first(genus),
|
||||
species = first(species),
|
||||
subspecies = first(subspecies),
|
||||
first_release = tags[min(release_order)],
|
||||
last_release = tags[max(release_order)],
|
||||
.groups = "drop"
|
||||
) %>%
|
||||
relocate(previous_names, .after = everything()) %>%
|
||||
arrange(mo)
|
||||
|
||||
message(
|
||||
nrow(registry), " MO codes, of which ", sum(registry$previous_names != ""),
|
||||
" had another name in an older release (check these when the registry changes)"
|
||||
)
|
||||
|
||||
write.csv(registry, registry_file, row.names = FALSE, na = "")
|
||||
message("Saved to ", registry_file)
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
"mo","registry_name","new_name","reason","approved_by","approved_date"
|
||||
"B_MYCBC_AVIM-C","Mycobacterium avium-intracellulare complex","Mycobacterium avium complex","Species group renamed in the development version of 2026; same group of organisms","",""
|
||||
|
Reference in new issue
Block a user