mirror of
https://github.com/msberends/AMR.git
synced 2026-11-15 22:20:47 +01:00
(v3.0.1.9096) MO code registry: codes never change meaning, are never guessed, and are validated
- data-raw/microorganisms_files/: registry of all MO codes of every release since v2.0.0 (78,992 codes, built from the git tags), approved renames, and README with the rules; decision by Matthijs S. Berends: MO codes from before v2.0.0 are not supported - as.mo(): codes of earlier releases are translated deterministically via the internal MO_RETIRED_CODES (generated from the registry); unknown codes are no longer guessed (previously F_TRCHP, Trichophyton in v3.0.1, became Talaromyces chlamydosporus) - synonym_mo_to_accepted_mo(): a missing target of one source no longer overwrites a valid target of another source (e.g. Chaetomium abuense), and no 'renamed to itself' - taxonomy build: codes are reused and reserved from the registry instead of the development data (fixes 40 codes reused in the 2026 development data), also for phylum to family; synonym cycles are broken; dangling 'renamed to' identifiers are removed; the build stops on any broken integrity rule before saving - run_microorganisms_build.R: runs the build in chunks at its checkpoints in one R process, with progress logging and resuming from any checkpoint; prompt updated accordingly - tests: shared integrity rules (helper-microorganisms.R) used by build and testthat, registry tests over all codes, sentinel organisms, regression tests; current defects of the data on main are listed in microorganisms_known_defects.csv until the next rebuild Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dr. M.S. (Matthijs) BerendsandClaude Opus 5.5 committed
1 parent
ca5fa333ca
commit
d683b9e7c6
16 files changed
+80516
-246
No files matched your search
+1
-1
@@ -1,5 +1,5 @@
|
||||
Package: AMR
|
||||
Version: 3.0.1.9095
|
||||
Version: 3.0.1.9096
|
||||
Date: 2026-10-03
|
||||
Title: Antimicrobial Resistance Data Analysis
|
||||
Description: Functions to simplify and standardise antimicrobial resistance (AMR)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# AMR 3.0.1.9095
|
||||
# AMR 3.0.1.9096
|
||||
|
||||
Planned as v3.1.0, mid October 2026.
|
||||
|
||||
@@ -24,6 +24,7 @@ Planned as v3.1.0, mid October 2026.
|
||||
Thus, `mo_domain()` was previously an alias of `mo_kingdom()`; it is now a distinct function returning the domain. Output of `mo_domain()` is therefore unchanged, while `mo_kingdom()` now returns the formal, new kingdom.
|
||||
|
||||
* Faster parallel computing via the `future` package for `as.sir()` and `wisca()`: a non-sequential plan (e.g. `future::plan(future::multisession)`) must be active before using `parallel = TRUE`.
|
||||
* `as.mo()` no longer guesses unknown MO codes (these become `NA` with a warning), and MO codes from versions before v2.0.0 are no longer supported
|
||||
|
||||
### New
|
||||
* EUCAST 2026 and CLSI 2026 breakpoints: over 5,700 new breakpoints added to the `clinical_breakpoints` data set; EUCAST 2026 is now the default for all MIC and disk diffusion interpretations
|
||||
@@ -41,6 +42,7 @@ Planned as v3.1.0, mid October 2026.
|
||||
* New `amr_course()` to download and unpack course or webinar materials from GitHub in one call
|
||||
* Typed missing value constants `NA_ab_` and `NA_mo_`, for use in pipelines that need missing values of a specific class
|
||||
* New `wisca_plot()` to assess the susceptibility and incidence distributions from the Monte Carlo simulations
|
||||
* MO codes of all releases since v2.0.0 are now reliably translated to their current code, also for taxa that moved to another domain; a registry guarantees that an MO code never gets another meaning
|
||||
|
||||
### Fixed
|
||||
* Streptomycin breakpoints imported from WHONET were coded as streptoduocin (`STR`) instead of streptomycin (`STR1`)
|
||||
@@ -62,6 +64,7 @@ Planned as v3.1.0, mid October 2026.
|
||||
* Translation fixes for Italian CoNS/CoPS names (#256), Dutch antimicrobials, and `sir_df()` foreign-language output (#272)
|
||||
* Fixed some EUCAST Expert Rules, mostly on *S. pneumoniae*
|
||||
* Logical comparison (`==`, `!=`) of `sir` class with a `factor` is now possible on R >= 4.3
|
||||
* `mo_current()` and `as.mo()` could not find the current name of some synonyms with multiple sources (e.g. *Chaetomium abuense*)
|
||||
|
||||
### Updated
|
||||
* `top_n_microorganisms()`: new `property_for_each` argument for sub-grouping within top *n* groups; rank ordering enforced (only lower taxonomic ranks allowed); fixed `property = NULL` not being accepted; inner filter now tracks original row indices to prevent cross-group contamination
|
||||
|
||||
@@ -1254,68 +1254,66 @@ parse_and_convert <- function(x) {
|
||||
}
|
||||
|
||||
replace_old_mo_codes <- function(x, property) {
|
||||
# this function transform old MO codes to current codes, such as:
|
||||
# B_ESCH_COL (AMR v0.5.0) -> B_ESCHR_COLI
|
||||
ind <- x %like_case% "^[A-Z]_[A-Z_]+$" & !x %in% AMR_env$MO_lookup$mo
|
||||
if (any(ind, na.rm = TRUE)) {
|
||||
add_MO_lookup_to_AMR_env()
|
||||
# get the ones that match
|
||||
affected <- x[ind]
|
||||
affected_unique <- unique(affected)
|
||||
all_direct_matches <- TRUE
|
||||
# find their new codes, once per code
|
||||
solved_unique <- unlist(lapply(
|
||||
strsplit(affected_unique, ""),
|
||||
function(m) {
|
||||
domain <- paste0("^", m[1])
|
||||
name <- m[3:length(m)]
|
||||
name[name == "_"] <- " "
|
||||
name <- tolower(paste0(name, ".*", collapse = ""))
|
||||
name <- gsub(" .*", " ", name, fixed = TRUE)
|
||||
name <- paste0("^", name)
|
||||
results <- AMR_env$MO_lookup$mo[AMR_env$MO_lookup$domain %like_case% domain &
|
||||
AMR_env$MO_lookup$fullname_lower %like_case% name]
|
||||
if (length(results) > 1) {
|
||||
all_direct_matches <<- FALSE
|
||||
}
|
||||
results[1L]
|
||||
}
|
||||
), use.names = FALSE)
|
||||
solved <- solved_unique[match(affected, affected_unique)]
|
||||
# assign on places where a match was found
|
||||
x[ind] <- solved
|
||||
n_matched <- length(affected[!is.na(affected)])
|
||||
n_solved <- length(affected[!is.na(solved)])
|
||||
n_unsolved <- length(affected[is.na(solved)])
|
||||
n_unique <- length(affected_unique[!is.na(affected_unique)])
|
||||
if (n_unique < n_matched) {
|
||||
n_unique <- paste0(n_unique, " unique, ")
|
||||
} else {
|
||||
n_unique <- ""
|
||||
# MO codes of earlier releases (since v2.0.0) that are not in the current data set anymore are translated
|
||||
# with the MO code registry (internal data `MO_RETIRED_CODES`, see data-raw/microorganisms_files/README.md),
|
||||
# e.g. a taxon that moved to another domain.
|
||||
# MO codes are never guessed: a wrong organism is worse than NA. Codes from releases before v2.0.0 are
|
||||
# not supported (decision by Matthijs S. Berends, 3 October 2026).
|
||||
add_MO_lookup_to_AMR_env()
|
||||
not_current <- !is.na(x) & !x %in% AMR_env$MO_lookup$mo
|
||||
if (!any(not_current)) {
|
||||
return(x)
|
||||
}
|
||||
# list at most 10 items, and escape curly brackets of names such as "Kapabacteria {class}" for the cli package
|
||||
listing <- function(v) {
|
||||
v <- gsub("}", "}}", gsub("{", "{{", v, fixed = TRUE), fixed = TRUE)
|
||||
out <- vector_and(utils::head(v, 10), quotes = FALSE, sort = FALSE)
|
||||
if (length(v) > 10) {
|
||||
out <- paste0(out, ", and ", length(v) - 10, " more")
|
||||
}
|
||||
if (property != "mo") {
|
||||
warning_(
|
||||
"in {.help [{.fun mo_", property, "}](AMR::mo_", property, ")}: the input contained ", n_matched,
|
||||
" old MO code", ifelse(n_matched == 1, "", "s"),
|
||||
" (", n_unique, "from another AMR package version). ",
|
||||
"Please update your MO codes with {.help [{.fun as.mo}](AMR::as.mo)} to increase speed."
|
||||
)
|
||||
} else {
|
||||
warning_(
|
||||
"in {.help [{.fun as.mo}](AMR::as.mo)}: the input contained ", n_matched,
|
||||
" old MO code", ifelse(n_matched == 1, "", "s"),
|
||||
" (", n_unique, "from another AMR package version). ",
|
||||
n_solved, " old MO code", ifelse(n_solved == 1, "", "s"),
|
||||
ifelse(n_solved == 1, " was", " were"),
|
||||
ifelse(all_direct_matches, " updated ", font_bold(" guessed ")),
|
||||
"to ", ifelse(n_solved == 1, "a ", ""),
|
||||
"currently used MO code", ifelse(n_solved == 1, "", "s"),
|
||||
ifelse(n_unsolved > 0,
|
||||
paste0(" and ", n_unsolved, " old MO code", ifelse(n_unsolved == 1, "", "s"), " could not be updated."),
|
||||
"."
|
||||
)
|
||||
out
|
||||
}
|
||||
in_function <- ifelse(property == "mo",
|
||||
"in {.help [{.fun as.mo}](AMR::as.mo)}: ",
|
||||
paste0("in {.help [{.fun mo_", property, "}](AMR::mo_", property, ")}: ")
|
||||
)
|
||||
|
||||
# codes from earlier releases
|
||||
is_retired <- not_current & x %in% MO_RETIRED_CODES$old_mo
|
||||
if (any(is_retired)) {
|
||||
retired <- unique(x[is_retired])
|
||||
current <- MO_RETIRED_CODES$mo[match(retired, MO_RETIRED_CODES$old_mo)]
|
||||
x[is_retired] <- current[match(x[is_retired], retired)]
|
||||
msg <- paste0(
|
||||
in_function, "the input contained ", length(retired), " MO code", ifelse(length(retired) == 1, "", "s"),
|
||||
" from an earlier AMR package version. "
|
||||
)
|
||||
if (any(!is.na(current))) {
|
||||
msg <- paste0(
|
||||
msg, sum(!is.na(current)), ifelse(sum(!is.na(current)) == 1, " was", " were"),
|
||||
" translated to the current MO code of the same taxon (", listing(paste0(retired[!is.na(current)], " -> ", current[!is.na(current)])), "). "
|
||||
)
|
||||
}
|
||||
if (any(is.na(current))) {
|
||||
gone <- MO_RETIRED_CODES$fullname[match(retired[is.na(current)], MO_RETIRED_CODES$old_mo)]
|
||||
msg <- paste0(
|
||||
msg, sum(is.na(current)), ifelse(sum(is.na(current)) == 1, " was", " were"),
|
||||
" set to NA, as the taxon is not in the current data set anymore (", listing(paste0(retired[is.na(current)], " = ", gone)), "). "
|
||||
)
|
||||
}
|
||||
warning_(msg, "Please update your MO codes with {.help [{.fun as.mo}](AMR::as.mo)}.")
|
||||
}
|
||||
|
||||
# anything else that looks like an MO code, but was never part of a release since v2.0.0
|
||||
is_unknown_code <- not_current & !is_retired & x %like_case% "^[A-Z]{1,2}_[][A-Z0-9_-]+$"
|
||||
if (any(is_unknown_code)) {
|
||||
unknown <- unique(x[is_unknown_code])
|
||||
x[is_unknown_code] <- NA_character_
|
||||
warning_(
|
||||
in_function, "the input contained ", length(unknown), " unknown MO code", ifelse(length(unknown) == 1, "", "s"),
|
||||
" (", listing(unknown), "), which ", ifelse(length(unknown) == 1, "was", "were"),
|
||||
" set to NA. MO codes of AMR package versions before v2.0.0 are not supported."
|
||||
)
|
||||
}
|
||||
x
|
||||
}
|
||||
@@ -1395,15 +1393,24 @@ synonym_mo_to_accepted_mo <- function(x, fill_in_accepted = FALSE, dataset = AMR
|
||||
x_mycobank <- dataset$mycobank_renamed_to[match(out, dataset$mo)]
|
||||
x_lpsn <- dataset$lpsn_renamed_to[match(out, dataset$mo)]
|
||||
|
||||
out[must_be_corrected & !is.na(x_gbif)] <- dataset$mo[match(x_gbif[must_be_corrected & !is.na(x_gbif)], dataset$gbif)]
|
||||
out[must_be_corrected & !is.na(x_mycobank)] <- dataset$mo[match(x_mycobank[must_be_corrected & !is.na(x_mycobank)], dataset$mycobank)]
|
||||
out[must_be_corrected & !is.na(x_lpsn)] <- dataset$mo[match(x_lpsn[must_be_corrected & !is.na(x_lpsn)], dataset$lpsn)]
|
||||
# priority LPSN > MycoBank > GBIF, but a source may only overwrite the result if its target exists in the
|
||||
# data set (before 3.1.0, e.g. a MycoBank target that was not included replaced a valid GBIF target by NA)
|
||||
by_gbif <- dataset$mo[match(x_gbif, dataset$gbif, incomparables = NA)]
|
||||
by_mycobank <- dataset$mo[match(x_mycobank, dataset$mycobank, incomparables = NA)]
|
||||
by_lpsn <- dataset$mo[match(x_lpsn, dataset$lpsn, incomparables = NA)]
|
||||
out[must_be_corrected & !is.na(by_gbif)] <- by_gbif[must_be_corrected & !is.na(by_gbif)]
|
||||
out[must_be_corrected & !is.na(by_mycobank)] <- by_mycobank[must_be_corrected & !is.na(by_mycobank)]
|
||||
out[must_be_corrected & !is.na(by_lpsn)] <- by_lpsn[must_be_corrected & !is.na(by_lpsn)]
|
||||
|
||||
is_still_synonym <- dataset$status[match(out, dataset$mo)] == "synonym"
|
||||
}
|
||||
|
||||
x_no_synonym <- dataset$status[match(x, dataset$mo)] != "synonym"
|
||||
out[x_no_synonym] <- NA_character_
|
||||
if (isFALSE(fill_in_accepted)) {
|
||||
# a synonym without any known current name is not 'renamed' to itself
|
||||
out[!is.na(out) & out == x] <- NA_character_
|
||||
}
|
||||
if (isTRUE(fill_in_accepted)) {
|
||||
out[!is.na(x_no_synonym) & x_no_synonym] <- x[!is.na(x_no_synonym) & x_no_synonym]
|
||||
}
|
||||
|
||||
Binary file not shown.
@@ -624,6 +624,28 @@ usethis::use_data(antimicrobials, overwrite = TRUE, version = 2, compress = "xz"
|
||||
pre_commit_lst$AB_LOOKUP <- create_AB_AV_lookup(antimicrobials)
|
||||
pre_commit_lst$AV_LOOKUP <- create_AB_AV_lookup(antivirals)
|
||||
|
||||
# MO codes of earlier releases that are not in the current data set anymore, with the current code of the same
|
||||
# taxon (see data-raw/microorganisms_files/README.md). as.mo() uses this to translate them, instead of guessing.
|
||||
source("tests/testthat/helper-microorganisms.R")
|
||||
mo_registry <- read_mo_registry(".")
|
||||
mo_renames <- read_mo_renames(".")
|
||||
retired <- mo_registry[!mo_registry$mo %in% as.character(microorganisms$mo), c("mo", "fullname", "rank"), drop = FALSE]
|
||||
renamed <- match(retired$mo, mo_renames$mo)
|
||||
retired$fullname[!is.na(renamed)] <- mo_renames$new_name[renamed[!is.na(renamed)]]
|
||||
# the same name, or else the same name without a rank suffix (such as "Kapabacteria {class}") and the same rank
|
||||
current_mo <- as.character(microorganisms$mo)[match(retired$fullname, microorganisms$fullname)]
|
||||
by_rank <- as.character(microorganisms$mo)[match(
|
||||
paste(mo_name_without_suffix(retired$fullname), retired$rank),
|
||||
paste(mo_name_without_suffix(microorganisms$fullname), microorganisms$rank)
|
||||
)]
|
||||
pre_commit_lst$MO_RETIRED_CODES <- data.frame(
|
||||
old_mo = retired$mo,
|
||||
mo = ifelse(is.na(current_mo), by_rank, current_mo),
|
||||
fullname = retired$fullname,
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
rm(mo_registry, mo_renames, retired, renamed, current_mo, by_rank)
|
||||
|
||||
# Export to package as internal data ----
|
||||
# usethis::use_data() must receive unquoted object names, which is not flexible at all.
|
||||
# we'll use good old base::save() instead
|
||||
|
||||
@@ -115,6 +115,9 @@ devtools::load_all(".") # to load the AMR package
|
||||
# keep the previous data set at hand, `AMR::microorganisms` will be replaced at the end of this script
|
||||
microorganisms_old <- AMR::microorganisms
|
||||
|
||||
# the integrity rules of the data set and the MO code registry, shared with the unit tests
|
||||
source("tests/testthat/helper-microorganisms.R")
|
||||
|
||||
|
||||
# Helper functions --------------------------------------------------------------------------------
|
||||
|
||||
@@ -143,6 +146,11 @@ review <- function(x, title = deparse(substitute(x))) {
|
||||
invisible(x)
|
||||
}
|
||||
|
||||
# taxonomic names in ASCII and without quotes
|
||||
ascii_names <- function(x) {
|
||||
gsub("[\"'`]", "", stringi::stri_trans_general(x, "Latin-ASCII"))
|
||||
}
|
||||
|
||||
# the priority of sources, used everywhere in this script
|
||||
source_priority <- c("LPSN" = 1L, "MycoBank" = 2L, "GBIF" = 3L, "manually added" = 4L, "inferred" = 5L)
|
||||
source_prio <- function(source) {
|
||||
@@ -1052,8 +1060,11 @@ saveRDS(taxonomy_mycobank, "data-raw/taxonomy_mycobank.rds", version = 2)
|
||||
|
||||
# Read GBIF data ----------------------------------------------------------------------------------
|
||||
|
||||
# @resume-block gbif_raw after=taxonomy_gbif
|
||||
# (needed by `current_gbif` when resuming from a checkpoint with run_microorganisms_build.R)
|
||||
taxonomy_gbif.bak <- vroom(file_gbif, guess_max = 5e5)
|
||||
colnames(taxonomy_gbif.bak) <- gsub(".*:(.*)", "\\1", colnames(taxonomy_gbif.bak))
|
||||
# @end-resume-block
|
||||
|
||||
# include all fungal orders from the mycobank db
|
||||
include_fungal_orders <- unique(taxonomy_mycobank$order[
|
||||
@@ -1282,10 +1293,7 @@ taxonomy_gbif <- readRDS("data-raw/taxonomy_gbif.rds")
|
||||
# Add full names ----------------------------------------------------------------------------------
|
||||
|
||||
# names must be ASCII before they are made unique, otherwise e.g. 'Fusarium aloës' and 'Fusarium aloes'
|
||||
# both end up as 'Fusarium aloes' in the final data set (which happened to 12 names in 2026)
|
||||
ascii_names <- function(x) {
|
||||
gsub("[\"'`]", "", stringi::stri_trans_general(x, "Latin-ASCII"))
|
||||
}
|
||||
# both end up as 'Fusarium aloes' in the final data set (which happened to 12 names in 2026), see ascii_names()
|
||||
taxonomy_gbif <- taxonomy_gbif %>%
|
||||
mutate(across(domain:subspecies, ascii_names))
|
||||
taxonomy_lpsn <- taxonomy_lpsn %>%
|
||||
@@ -1639,6 +1647,8 @@ saveRDS(taxonomy, "data-raw/taxonomy0.rds")
|
||||
# Ensure every referenced rank has its own row (domain through species).
|
||||
# Where possible, enrich with GBIF identifiers.
|
||||
|
||||
# @resume-block current_gbif after=taxonomy1
|
||||
# (needed by add_missing_parents() after filtering, when resuming from a checkpoint)
|
||||
current_gbif <- taxonomy_gbif.bak %>%
|
||||
filter(is.na(acceptedNameUsageID)) %>%
|
||||
mutate(
|
||||
@@ -1663,6 +1673,7 @@ current_gbif <- taxonomy_gbif.bak %>%
|
||||
phylum = if_else(taxonRank == "phylum", canonical, phylum)
|
||||
) %>%
|
||||
select(-canonical)
|
||||
# @end-resume-block
|
||||
|
||||
taxonomy <- add_missing_parents(taxonomy, current_gbif)
|
||||
|
||||
@@ -2035,6 +2046,8 @@ taxonomy$rank[is.na(taxonomy$rank)] <- "domain"
|
||||
# this part is required here, because it's needed for filtering on 'relevant' species to keep later on
|
||||
# (`pathogens` was read at the start of this script)
|
||||
|
||||
# @resume-block prevalence after=taxonomy2
|
||||
# (compute_prevalence() is used again later on, when resuming from a checkpoint)
|
||||
# get all established, both old and current taxonomic names
|
||||
established <- pathogens %>%
|
||||
filter(status == "established") %>%
|
||||
@@ -2147,6 +2160,7 @@ compute_prevalence <- function(taxonomy) {
|
||||
mutate(prevalence = if_else(rank == "genus", pmin(prevalence, gen_prevalence, na.rm = TRUE), prevalence)) %>%
|
||||
select(-gen_prevalence)
|
||||
}
|
||||
# @end-resume-block
|
||||
taxonomy <- compute_prevalence(taxonomy)
|
||||
|
||||
table(taxonomy$prevalence, useNA = "always")
|
||||
@@ -2513,6 +2527,77 @@ taxonomy <- taxonomy %>%
|
||||
distinct(fullname, .keep_all = TRUE)
|
||||
message("Removed ", n_before - nrow(taxonomy), " nonsense names")
|
||||
|
||||
# Break cycles of synonyms ------------------------------------------------------------------------
|
||||
|
||||
# Sources can contradict each other, e.g. in 2026 GBIF listed Capillidium denaeosporus as synonym of
|
||||
# Conidiobolus denaeosporum and also the other way around, so that the 'current name' depended on the starting
|
||||
# point. In a cycle, the statement of the most authoritative source counts: that record stays a synonym, and its
|
||||
# target becomes accepted.
|
||||
next_synonym_step <- function(df) {
|
||||
by_gbif <- df$fullname[match(df$gbif_renamed_to, df$gbif, incomparables = NA)]
|
||||
by_mycobank <- df$fullname[match(df$mycobank_renamed_to, df$mycobank, incomparables = NA)]
|
||||
by_lpsn <- df$fullname[match(df$lpsn_renamed_to, df$lpsn, incomparables = NA)]
|
||||
out <- coalesce(by_lpsn, by_mycobank, by_gbif)
|
||||
out[df$status != "synonym"] <- NA_character_
|
||||
setNames(out, df$fullname)
|
||||
}
|
||||
synonym_next <- next_synonym_step(taxonomy)
|
||||
synonym_cycles <- list()
|
||||
for (start in names(synonym_next)[!is.na(synonym_next)]) {
|
||||
path <- start
|
||||
repeat {
|
||||
nxt <- unname(synonym_next[path[length(path)]])
|
||||
if (is.na(nxt)) {
|
||||
break
|
||||
}
|
||||
if (nxt %in% path) {
|
||||
cycle <- sort(path[match(nxt, path):length(path)])
|
||||
synonym_cycles[[paste(cycle, collapse = " | ")]] <- cycle
|
||||
break
|
||||
}
|
||||
path <- c(path, nxt)
|
||||
if (length(path) > 25) {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
# If multiple records in the cycle come from the most authoritative source, the sources contradict themselves and
|
||||
# the direction cannot be determined automatically (in 2026, even the authors in `ref` of these GBIF records were
|
||||
# unreliable). Then ALL records in the cycle become accepted, so that no wrong current name is given, and they are
|
||||
# shown for a human decision.
|
||||
cycle_fixes <- bind_rows(lapply(synonym_cycles, function(cycle_members) {
|
||||
# (not named `cycle`, since tibble() below would then use its own new column `cycle`)
|
||||
members <- taxonomy[match(cycle_members, taxonomy$fullname), , drop = FALSE]
|
||||
best <- members[source_prio(members$source) == min(source_prio(members$source)), , drop = FALSE]
|
||||
if (nrow(best) == 1) {
|
||||
tibble(
|
||||
cycle = paste(cycle_members, collapse = " | "),
|
||||
made_accepted = unname(synonym_next[best$fullname]),
|
||||
reason = paste("statement of", best$fullname, "from", best$source, "(most authoritative source)")
|
||||
)
|
||||
} else {
|
||||
tibble(
|
||||
cycle = paste(cycle_members, collapse = " | "),
|
||||
made_accepted = cycle_members,
|
||||
reason = paste("contradicting statements of", toString(unique(best$source)), "- all accepted, NEEDS A HUMAN DECISION")
|
||||
)
|
||||
}
|
||||
}))
|
||||
review(cycle_fixes, "Cycles of synonyms")
|
||||
if (nrow(cycle_fixes) > 0) {
|
||||
taxonomy <- taxonomy %>%
|
||||
mutate(
|
||||
fix = fullname %in% cycle_fixes$made_accepted,
|
||||
status = if_else(fix, "accepted", status),
|
||||
lpsn_renamed_to = if_else(fix, NA_character_, lpsn_renamed_to),
|
||||
mycobank_renamed_to = if_else(fix, NA_character_, mycobank_renamed_to),
|
||||
gbif_renamed_to = if_else(fix, NA_character_, gbif_renamed_to)
|
||||
) %>%
|
||||
select(-fix)
|
||||
}
|
||||
rm(synonym_next, synonym_cycles, cycle_fixes)
|
||||
|
||||
|
||||
# Fix genera that are synonyms while they contain accepted species --------------------------------
|
||||
|
||||
# e.g. in 2026, MycoBank listed the genus Blastomyces as a synonym, while Blastomyces dermatitidis was accepted
|
||||
@@ -2560,151 +2645,59 @@ mo_domain <- taxonomy %>%
|
||||
)
|
||||
mo_domain
|
||||
|
||||
# `domain` exists in the data set since 2026, so codes are kept per domain
|
||||
# (in the 2026 build, this was `AMR::microorganisms %>% rename(domain = kingdom)`, which would now fail)
|
||||
existing_mo_tbl <- microorganisms_old %>%
|
||||
mutate(mo = as.character(mo))
|
||||
|
||||
# @resume-block mo_registry after=taxonomy2c
|
||||
# (the MO code registry is used by the final checks, when resuming from a checkpoint)
|
||||
# parts of existing MO codes, split on the underscore (genus codes can contain digits or hyphens, such as
|
||||
# B_RBCLM1 or B_CND-P, so a regex with only [A-Z] does not work and caused repeated elements like
|
||||
# B_SCLLM_CNNM_LNSM_LNSM_LNSM in earlier versions)
|
||||
mo_part <- function(mo, i) {
|
||||
vapply(strsplit(as.character(mo), "_", fixed = TRUE), function(x) if (length(x) >= i) x[i] else NA_character_, character(1))
|
||||
}
|
||||
# parts of a name, e.g. the species of 'Escherichia coli'
|
||||
mo_part_of_name <- function(x, i) {
|
||||
vapply(strsplit(x, " ", fixed = TRUE), function(x) if (length(x) >= i) x[i] else "", character(1))
|
||||
}
|
||||
|
||||
# Existing codes come from the MO code registry (data-raw/microorganisms_files/), which contains every code of
|
||||
# every release since v2.0.0, and NOT from the previous data set: codes that only existed in a development
|
||||
# version carry no weight (decision by Matthijs S. Berends, 3 October 2026). A registered code is never given
|
||||
# to another taxon, also not when its taxon is not in the new data set anymore, since users store these codes.
|
||||
mo_registry <- read_mo_registry(".")
|
||||
mo_renames <- read_mo_renames(".")
|
||||
if (is.null(mo_registry)) {
|
||||
stop("MO code registry not found, see data-raw/microorganisms_files/README.md", call. = FALSE)
|
||||
}
|
||||
release_order <- function(release) {
|
||||
as.integer(factor(release, levels = unique(release[order(numeric_version(sub("^v", "", release)))])))
|
||||
}
|
||||
existing_mo_tbl <- mo_registry %>%
|
||||
# approved renames, e.g. a corrected spelling: the code continues with the new name
|
||||
left_join(mo_renames %>% select(mo, new_name), by = "mo") %>%
|
||||
mutate(
|
||||
fullname = coalesce(new_name, fullname),
|
||||
species = if_else(!is.na(new_name) & rank %in% c("species", "subspecies"), mo_part_of_name(new_name, 2), species),
|
||||
subspecies = if_else(!is.na(new_name) & rank == "subspecies", mo_part_of_name(new_name, 3), subspecies)
|
||||
) %>%
|
||||
select(-new_name) %>%
|
||||
# if a taxon had multiple codes over time, the code of the most recent release is used
|
||||
arrange(desc(release_order(last_release)), desc(release_order(first_release))) %>%
|
||||
distinct(domain, rank, fullname, .keep_all = TRUE)
|
||||
|
||||
# all codes that have ever been used must never be given to another taxon, also not when their taxon
|
||||
# is not in the new data set anymore (users may have stored old codes)
|
||||
reserved_genus_codes <- existing_mo_tbl %>%
|
||||
reserved_genus_codes <- mo_registry %>%
|
||||
filter(rank %in% c("genus", "species", "subspecies")) %>%
|
||||
transmute(domain, code = mo_part(mo, 2)) %>%
|
||||
distinct()
|
||||
reserved_species_codes <- existing_mo_tbl %>%
|
||||
reserved_species_codes <- mo_registry %>%
|
||||
filter(rank %in% c("species", "subspecies")) %>%
|
||||
transmute(domain, genus_code = mo_part(mo, 2), code = mo_part(mo, 3)) %>%
|
||||
distinct()
|
||||
# @end-resume-block
|
||||
|
||||
|
||||
# phylum until family are abbreviated with 8 characters and prefixed with their rank
|
||||
|
||||
# Phylum - keep old and fill up for new ones
|
||||
mo_phylum <- taxonomy %>%
|
||||
filter(rank == "phylum") %>%
|
||||
distinct(domain, phylum) %>%
|
||||
left_join(
|
||||
existing_mo_tbl %>%
|
||||
filter(rank == "phylum") %>%
|
||||
transmute(
|
||||
domain,
|
||||
phylum = fullname,
|
||||
mo_old = gsub("^[A-Z]{1,2}_", "", as.character(mo))
|
||||
),
|
||||
by = c("domain", "phylum")
|
||||
) %>%
|
||||
group_by(domain) %>%
|
||||
mutate(
|
||||
mo_phylum8 = AMR:::abbreviate_mo(phylum, minlength = 8, prefix = "[PHL]_"),
|
||||
mo_phylum9 = AMR:::abbreviate_mo(phylum, minlength = 9, prefix = "[PHL]_"),
|
||||
mo_phylum = if_else(!is.na(mo_old), mo_old, mo_phylum8),
|
||||
mo_duplicated = duplicated(mo_phylum),
|
||||
mo_phylum = if_else(mo_duplicated, mo_phylum9, mo_phylum),
|
||||
mo_duplicated = duplicated(mo_phylum)
|
||||
) %>%
|
||||
ungroup()
|
||||
if (any(mo_phylum$mo_duplicated, na.rm = TRUE)) {
|
||||
stop("Duplicate MO codes for phylum!")
|
||||
}
|
||||
mo_phylum <- mo_phylum %>%
|
||||
select(domain, phylum, mo_phylum)
|
||||
|
||||
# Class - keep old and fill up for new ones
|
||||
mo_class <- taxonomy %>%
|
||||
filter(rank == "class") %>%
|
||||
distinct(domain, class) %>%
|
||||
left_join(
|
||||
existing_mo_tbl %>%
|
||||
filter(rank == "class") %>%
|
||||
transmute(
|
||||
domain,
|
||||
class = fullname,
|
||||
mo_old = gsub("^[A-Z]{1,2}_", "", as.character(mo))
|
||||
),
|
||||
by = c("domain", "class")
|
||||
) %>%
|
||||
group_by(domain) %>%
|
||||
mutate(
|
||||
mo_class8 = AMR:::abbreviate_mo(class, minlength = 8, prefix = "[CLS]_"),
|
||||
mo_class9 = AMR:::abbreviate_mo(class, minlength = 9, prefix = "[CLS]_"),
|
||||
mo_class = if_else(!is.na(mo_old), mo_old, mo_class8),
|
||||
mo_duplicated = duplicated(mo_class),
|
||||
mo_class = if_else(mo_duplicated, mo_class9, mo_class),
|
||||
mo_duplicated = duplicated(mo_class)
|
||||
) %>%
|
||||
ungroup()
|
||||
if (any(mo_class$mo_duplicated, na.rm = TRUE)) {
|
||||
stop("Duplicate MO codes for class!")
|
||||
}
|
||||
mo_class <- mo_class %>%
|
||||
select(domain, class, mo_class)
|
||||
|
||||
# Order - keep old and fill up for new ones
|
||||
mo_order <- taxonomy %>%
|
||||
filter(rank == "order") %>%
|
||||
distinct(domain, order) %>%
|
||||
left_join(
|
||||
existing_mo_tbl %>%
|
||||
filter(rank == "order") %>%
|
||||
transmute(
|
||||
domain,
|
||||
order = fullname,
|
||||
mo_old = gsub("^[A-Z]{1,2}_", "", as.character(mo))
|
||||
),
|
||||
by = c("domain", "order")
|
||||
) %>%
|
||||
group_by(domain) %>%
|
||||
mutate(
|
||||
mo_order8 = AMR:::abbreviate_mo(order, minlength = 8, prefix = "[ORD]_"),
|
||||
mo_order9 = AMR:::abbreviate_mo(order, minlength = 9, prefix = "[ORD]_"),
|
||||
mo_order = if_else(!is.na(mo_old), mo_old, mo_order8),
|
||||
mo_duplicated = duplicated(mo_order),
|
||||
mo_order = if_else(mo_duplicated, mo_order9, mo_order),
|
||||
mo_duplicated = duplicated(mo_order)
|
||||
) %>%
|
||||
ungroup()
|
||||
if (any(mo_order$mo_duplicated, na.rm = TRUE)) {
|
||||
stop("Duplicate MO codes for order!")
|
||||
}
|
||||
mo_order <- mo_order %>%
|
||||
select(domain, order, mo_order)
|
||||
|
||||
# Family - keep old and fill up for new ones
|
||||
mo_family <- taxonomy %>%
|
||||
filter(rank == "family") %>%
|
||||
distinct(domain, family) %>%
|
||||
left_join(
|
||||
existing_mo_tbl %>%
|
||||
filter(rank == "family") %>%
|
||||
transmute(
|
||||
domain,
|
||||
family = fullname,
|
||||
mo_old = gsub("^[A-Z]{1,2}_", "", as.character(mo))
|
||||
),
|
||||
by = c("domain", "family")
|
||||
) %>%
|
||||
group_by(domain) %>%
|
||||
mutate(
|
||||
mo_family8 = AMR:::abbreviate_mo(family, minlength = 8, prefix = "[FAM]_"),
|
||||
mo_family9 = AMR:::abbreviate_mo(family, minlength = 9, prefix = "[FAM]_"),
|
||||
mo_family = if_else(!is.na(mo_old), mo_old, mo_family8),
|
||||
mo_duplicated = duplicated(mo_family),
|
||||
mo_family = if_else(mo_duplicated, mo_family9, mo_family),
|
||||
mo_duplicated = duplicated(mo_family)
|
||||
) %>%
|
||||
ungroup()
|
||||
if (any(mo_family$mo_duplicated, na.rm = TRUE)) {
|
||||
stop("Duplicate MO codes for family!")
|
||||
}
|
||||
mo_family <- mo_family %>%
|
||||
select(domain, family, mo_family)
|
||||
|
||||
# keep the old code where available, and give every new taxon the first candidate code that is not in use
|
||||
# and was never used before (`reserved`) - if all candidates are taken, a number is added to the first one
|
||||
assign_codes <- function(old, candidates, reserved = character(0)) {
|
||||
@@ -2726,6 +2719,47 @@ assign_codes <- function(old, candidates, reserved = character(0)) {
|
||||
out
|
||||
}
|
||||
|
||||
# Phylum until family - keep old (from the registry) and fill up for new ones, never with a registered code
|
||||
higher_rank_codes <- function(rank_name, tag) {
|
||||
reserved <- mo_registry %>%
|
||||
filter(rank == rank_name) %>%
|
||||
transmute(domain, code = sub("^[A-Z]{1,2}_", "", mo))
|
||||
out <- taxonomy %>%
|
||||
filter(rank == rank_name) %>%
|
||||
distinct(domain, name = .data[[rank_name]]) %>%
|
||||
left_join(
|
||||
existing_mo_tbl %>%
|
||||
filter(rank == rank_name) %>%
|
||||
transmute(domain, name = mo_name_without_suffix(fullname), mo_old = sub("^[A-Z]{1,2}_", "", mo)) %>%
|
||||
distinct(domain, name, .keep_all = TRUE),
|
||||
by = c("domain", "name")
|
||||
) %>%
|
||||
group_by(domain) %>%
|
||||
mutate(
|
||||
code = assign_codes(
|
||||
old = mo_old,
|
||||
candidates = Map(
|
||||
c,
|
||||
AMR:::abbreviate_mo(name, minlength = 8, prefix = paste0("[", tag, "]_")),
|
||||
AMR:::abbreviate_mo(name, minlength = 9, prefix = paste0("[", tag, "]_"))
|
||||
),
|
||||
reserved = reserved$code[reserved$domain == cur_group()$domain]
|
||||
)
|
||||
) %>%
|
||||
ungroup()
|
||||
if (anyDuplicated(paste(out$domain, out$code)) > 0 || anyNA(out$code)) {
|
||||
stop("Duplicate MO codes for ", rank_name, "!", call. = FALSE)
|
||||
}
|
||||
out %>%
|
||||
select(domain, name, code) %>%
|
||||
setNames(c("domain", rank_name, paste0("mo_", rank_name)))
|
||||
}
|
||||
mo_phylum <- higher_rank_codes("phylum", "PHL")
|
||||
mo_class <- higher_rank_codes("class", "CLS")
|
||||
mo_order <- higher_rank_codes("order", "ORD")
|
||||
mo_family <- higher_rank_codes("family", "FAM")
|
||||
|
||||
|
||||
# construct code part for genus - keep old code where available and generate new ones where needed
|
||||
mo_genus <- taxonomy %>%
|
||||
filter(rank == "genus") %>%
|
||||
@@ -2863,6 +2897,13 @@ mo_subspecies <- mo_subspecies %>%
|
||||
mo_unknown <- existing_mo_tbl %>%
|
||||
filter(fullname %like% "unknown") %>%
|
||||
transmute(fullname, mo_unknown = as.character(mo))
|
||||
# unknowns that were added after the last release keep their code too, as long as it is not registered
|
||||
mo_unknown <- mo_unknown %>%
|
||||
bind_rows(
|
||||
microorganisms_old %>%
|
||||
filter(fullname %like% "unknown", !fullname %in% mo_unknown$fullname, !mo %in% mo_registry$mo) %>%
|
||||
transmute(fullname, mo_unknown = as.character(mo))
|
||||
)
|
||||
|
||||
# apply the new codes!
|
||||
taxonomy1 <- taxonomy
|
||||
@@ -3588,23 +3629,33 @@ microorganisms_old %>%
|
||||
select(mo, fullname, rank, status, prevalence) %>%
|
||||
review("Previously manually added taxa that are not in the new data set")
|
||||
|
||||
# LPSN records must be prokaryotes and MycoBank records must be fungi, also in their MO code (issue #309)
|
||||
if (!all(taxonomy$domain[taxonomy$source == "LPSN"] %in% c("Bacteria", "Archaea")) ||
|
||||
!all(taxonomy$mo[taxonomy$source == "LPSN"] %like% "^(B|A)_") ||
|
||||
!all(taxonomy$domain[taxonomy$source == "MycoBank"] == "Fungi") ||
|
||||
!all(taxonomy$mo[taxonomy$source == "MycoBank"] %like% "^F_")) {
|
||||
warning("LPSN records outside prokaryotes or MycoBank records outside fungi, check 'Combine the datasets'", call. = FALSE)
|
||||
}
|
||||
|
||||
# these must never fail, as the package relies on them
|
||||
stopifnot(
|
||||
"MO codes must be unique" = !anyDuplicated(taxonomy$mo),
|
||||
"full names must be unique" = !anyDuplicated(taxonomy$fullname),
|
||||
"MO codes must not be empty" = !any(taxonomy$mo %in% c("", NA)),
|
||||
"prevalence must not be empty" = !anyNA(taxonomy$prevalence),
|
||||
"status must be accepted, synonym or unknown" = all(taxonomy$status %in% c("accepted", "synonym", "unknown")),
|
||||
"sources must be known" = all(taxonomy$source %in% c("LPSN", "MycoBank", "GBIF", "manually added"))
|
||||
# 'renamed to' identifiers of records that were removed after the earlier clean-up (e.g. by deduplication) lead
|
||||
# nowhere, so remove them again: these synonyms then have no current name, instead of a broken one
|
||||
n_dangling <- sum(
|
||||
(!is.na(taxonomy$lpsn_renamed_to) & !taxonomy$lpsn_renamed_to %in% taxonomy$lpsn) |
|
||||
(!is.na(taxonomy$mycobank_renamed_to) & !taxonomy$mycobank_renamed_to %in% taxonomy$mycobank) |
|
||||
(!is.na(taxonomy$gbif_renamed_to) & !taxonomy$gbif_renamed_to %in% taxonomy$gbif)
|
||||
)
|
||||
taxonomy <- taxonomy %>%
|
||||
mutate(
|
||||
lpsn_renamed_to = if_else(lpsn_renamed_to %in% lpsn, lpsn_renamed_to, NA_character_),
|
||||
mycobank_renamed_to = if_else(mycobank_renamed_to %in% mycobank, mycobank_renamed_to, NA_character_),
|
||||
gbif_renamed_to = if_else(gbif_renamed_to %in% gbif, gbif_renamed_to, NA_character_)
|
||||
)
|
||||
message("Removed dangling 'renamed to' identifiers of ", n_dangling, " records")
|
||||
|
||||
# synonyms without a current name are no error, but every one of them is a name that as.mo() cannot update
|
||||
review(mo_synonyms_without_current_name(taxonomy), "Synonyms without a current name")
|
||||
|
||||
# the integrity rules of the data set and the MO code registry (shared with the unit tests in
|
||||
# tests/testthat/helper-microorganisms.R): these must never fail, as the package and its users rely on them
|
||||
integrity_issues <- mo_integrity_issues(taxonomy, registry = mo_registry, renames = mo_renames)
|
||||
if (!is.null(mo_integrity_report(integrity_issues))) {
|
||||
stop("The new data set breaks integrity rules, fix the cause in this script:\n",
|
||||
mo_integrity_report(integrity_issues),
|
||||
call. = FALSE
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
# Update other data sets --------------------------------------------------------------------------
|
||||
@@ -3676,11 +3727,13 @@ taxonomy <- taxonomy %>%
|
||||
AMR:::dataset_UTF8_to_ASCII()
|
||||
|
||||
# check again after the conversion to ASCII
|
||||
stopifnot(
|
||||
"full names must be unique after conversion to ASCII" = !anyDuplicated(taxonomy$fullname),
|
||||
"MO codes must be unique after conversion to ASCII" = !anyDuplicated(taxonomy$mo),
|
||||
"MO codes may only contain A-Z, 0-9, _, -, [ and ]" = all(taxonomy$mo %like_case% "^[][A-Z0-9_-]+$")
|
||||
)
|
||||
integrity_issues <- mo_integrity_issues(taxonomy, registry = mo_registry, renames = mo_renames)
|
||||
if (!is.null(mo_integrity_report(integrity_issues))) {
|
||||
stop("The data set breaks integrity rules after the conversion to ASCII:\n",
|
||||
mo_integrity_report(integrity_issues),
|
||||
call. = FALSE
|
||||
)
|
||||
}
|
||||
|
||||
microorganisms <- taxonomy
|
||||
|
||||
|
||||
@@ -65,20 +65,22 @@ These are needed for `TAXONOMY_VERSION` in `R/aa_globals.R` later. Write them do
|
||||
## Step 2: run the build
|
||||
|
||||
The script takes several hours (most of it is LPSN scraping, which is cached in
|
||||
`data-raw/lpsn_scrape_cache.rds`, so a restart is fast). Run it detached from your tool calls, so that
|
||||
it survives time limits, and write everything to a log:
|
||||
`data-raw/lpsn_scrape_cache.rds`, so a restart is fast). Never `source()` the whole script. Use the runner
|
||||
`data-raw/_reproduction_scripts/run_microorganisms_build.R`: it cuts the script into chunks at its
|
||||
checkpoints, runs them one by one in a single R process, logs each chunk with its duration in
|
||||
`<scratch>/logs/progress.tsv`, stops with the name of the failing chunk, and can resume from any checkpoint.
|
||||
See the chunks first with `--list`. Run it detached from your tool calls, so that it survives time limits:
|
||||
|
||||
```bash
|
||||
mkdir -p <scratch>/logs
|
||||
cd <repo root>
|
||||
nohup Rscript -e 'options(AMR_build_view = FALSE, AMR_build_print_rows = 200, AMR_build_review_dir = "<review dir>", warn = 1, width = 200); source("data-raw/_reproduction_scripts/reproduction_of_microorganisms.R", echo = TRUE, max.deparse.length = Inf, keep.source = TRUE)' > <scratch>/logs/build_1.log 2>&1 &
|
||||
nohup Rscript data-raw/_reproduction_scripts/run_microorganisms_build.R --review-dir=<review dir> --log-dir=<scratch>/logs > <scratch>/logs/build_1.log 2>&1 &
|
||||
```
|
||||
|
||||
Monitor the log at sensible intervals (every 10 to 20 minutes, not continuously). Read every
|
||||
`>>> REVIEW:` block as soon as it appears; you do not have to wait until the end to start thinking.
|
||||
With `AMR_build_review_dir` set, every review block is also saved in full as a CSV file in
|
||||
`<review dir>`, named after its title. These files are part of the PR. Set the same option in every
|
||||
resumed run.
|
||||
Monitor the log and `progress.tsv` at sensible intervals (every 10 to 20 minutes, not continuously). Read
|
||||
every `>>> REVIEW:` block as soon as it appears; you do not have to wait until the end to start thinking.
|
||||
Every review block is also saved in full as a CSV file in `<review dir>`, named after its title. These
|
||||
files are part of the PR.
|
||||
|
||||
### Intermediate results
|
||||
|
||||
@@ -105,24 +107,18 @@ and must not be committed.
|
||||
|
||||
### If the run fails
|
||||
|
||||
1. Read the error and the 100 lines before it. Find the root cause in the script, fix it, and
|
||||
check the syntax with `Rscript -e 'invisible(parse("<script>"))'`.
|
||||
2. Do not start from scratch if a checkpoint exists. Write a resume script in `<scratch>` (never in
|
||||
the repository) that contains, in this order:
|
||||
- the script from the top up to (not including) `# Read LPSN data ---`: setup, guard, helpers,
|
||||
clinically relevant genera;
|
||||
- `taxonomy_lpsn`, `taxonomy_mycobank` and `taxonomy_gbif` from their `.rds` files;
|
||||
- if resuming at or after `taxonomy0.rds`: the two lines that read the raw GBIF file
|
||||
(`taxonomy_gbif.bak <- vroom(...)` and the `colnames()` line after it) and the definition of
|
||||
`current_gbif` (needed by `add_missing_parents()`);
|
||||
- if resuming at or after `taxonomy2.rds`: the section `# Add prevalence ---` up to and including
|
||||
the definition of `compute_prevalence()` (needed later on);
|
||||
- `taxonomy <- readRDS("data-raw/<latest checkpoint>.rds")`;
|
||||
- the remainder of the script after that checkpoint's `saveRDS()` line.
|
||||
Number the logs (`build_2.log`, etc.).
|
||||
3. If the same section fails three times for different reasons, or the cause lies outside the
|
||||
script (e.g. the format of a source file changed fundamentally), stop and report (see "When to
|
||||
stop").
|
||||
1. Read the error in the log and the 100 lines before it, and `progress.tsv`. Find the root cause in
|
||||
the script, fix it, and check the syntax with `Rscript -e 'invisible(parse("<script>"))'`.
|
||||
2. Do not start from scratch if a checkpoint exists. The runner tells you where to resume; add
|
||||
`--resume-from=<checkpoint>` (e.g. `--resume-from=taxonomy1b`) and use the same `--review-dir` and
|
||||
`--log-dir`. It runs the setup chunk, loads the saved results and the parts of earlier chunks that
|
||||
later chunks need (the blocks marked with `# @resume-block` in the script), and continues after that
|
||||
checkpoint. Number the logs (`build_2.log`, etc.).
|
||||
3. If you add code to an earlier chunk that a later chunk needs (a function or an object), either move it
|
||||
to the setup part at the top of the script, or mark it as a `# @resume-block` with the right `after=`,
|
||||
otherwise resuming breaks. Check with `--list`.
|
||||
4. If the same chunk fails three times for different reasons, or the cause lies outside the script (e.g.
|
||||
the format of a source file changed fundamentally), stop and report (see "When to stop").
|
||||
|
||||
## Step 3: review as an expert
|
||||
|
||||
@@ -207,7 +203,19 @@ change of more than 5% in a domain needs an explanation. Also report:
|
||||
- the 30 most relevant (prevalence <= 1.25) new genera and species, and the 30 most relevant removed
|
||||
ones.
|
||||
|
||||
### 3d. Unit tests
|
||||
### 3d. Integrity rules and MO code registry
|
||||
|
||||
The script stops before saving if any rule in `tests/testthat/helper-microorganisms.R` is broken, including
|
||||
the MO code registry rules (see `data-raw/microorganisms_files/README.md`): a code from any release since
|
||||
v2.0.0 may never be given to another taxon. Never weaken a rule to get past this; fix the cause. A registered
|
||||
code may only get another name (e.g. a corrected spelling) if it is listed in
|
||||
`data-raw/microorganisms_files/mo_code_renames.csv`, and you may only add such rows with an empty
|
||||
`approved_by`: the build will then stop on it until the human approves it, so list every proposed rename under
|
||||
"Needs a human". After the build, `tests/testthat/microorganisms_known_defects.csv` (the known defects of the
|
||||
data set before this rebuild) must be emptied: delete the rows that are solved, and the file itself if none
|
||||
are left. Any remaining row must be explained under "Needs a human".
|
||||
|
||||
### 3e. Unit tests
|
||||
|
||||
After the data are saved and the package is reloaded, run the tests at the end of the script and then
|
||||
the full suite with `devtools::test()`. Analyse every failure: is the test outdated because of a
|
||||
@@ -242,7 +250,7 @@ human can review it before merging and next year's run can learn from it. Struct
|
||||
Every item states the check, the finding in numbers, your decision, and a link to the CSV file in
|
||||
`<review dir>` where relevant. There is one item for each review block of 3a, one for each sentinel
|
||||
organism of 3b that deviates (and one item for all sentinels that passed), one for each comparison
|
||||
of 3c, one for the unit tests (3d) and one for each changed test. Leave all boxes unticked: ticking
|
||||
of 3c, one for the integrity rules and registry (3d), one for the unit tests (3e) and one for each changed test. Leave all boxes unticked: ticking
|
||||
them is the human's job, it means "reviewed and agreed".
|
||||
5. **Needs a human**: also as checkboxes, everything you could not verify or decide, each with your
|
||||
recommendation. Be concrete, e.g. "COL lists *Enterobius vermicularis* as a synonym of ...;
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
# ==================================================================== #
|
||||
# TITLE: #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data #
|
||||
# #
|
||||
# SOURCE CODE: #
|
||||
# https://github.com/msberends/AMR #
|
||||
# #
|
||||
# PLEASE CITE THIS SOFTWARE AS: #
|
||||
# Berends MS, Luz CF, Friedrich AW, et al. (2022). #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data. #
|
||||
# Journal of Statistical Software, 104(3), 1-31. #
|
||||
# https://doi.org/10.18637/jss.v104.i03 #
|
||||
# #
|
||||
# Developed at the University of Groningen and the University Medical #
|
||||
# Center Groningen in the Netherlands, in collaboration with many #
|
||||
# colleagues from around the world, see our website. #
|
||||
# #
|
||||
# This R package is free software; you can freely use and distribute #
|
||||
# it for both personal and commercial purposes under the terms of the #
|
||||
# GNU General Public License version 2.0 (GNU GPL-2), as published by #
|
||||
# the Free Software Foundation. #
|
||||
# We created this package for both routine data analysis and academic #
|
||||
# research and it was publicly released in the hope that it will be #
|
||||
# useful, but it comes WITHOUT ANY WARRANTY OR LIABILITY. #
|
||||
# #
|
||||
# Visit our website for the full manual and a complete tutorial about #
|
||||
# how to conduct AMR data analysis: https://amr-for-r.org #
|
||||
# ==================================================================== #
|
||||
|
||||
# Runs reproduction_of_microorganisms.R non-interactively, in chunks, in one R process.
|
||||
#
|
||||
# The script itself stays one file (to be run line by line by a human as well). This runner cuts it into chunks
|
||||
# at its checkpoints (the saveRDS() lines of data-raw/taxonomy*.rds), runs these one by one, logs the start,
|
||||
# end and duration of each chunk in a progress file, and stops with the name of the failing chunk.
|
||||
# It can resume from any checkpoint: it then runs the setup chunk, loads the saved results and the parts of
|
||||
# earlier chunks that later chunks need (marked with '# @resume-block' in the script), and continues.
|
||||
#
|
||||
# Usage, from the root of the repository:
|
||||
# Rscript data-raw/_reproduction_scripts/run_microorganisms_build.R [options]
|
||||
# Options:
|
||||
# --review-dir=DIR folder for the CSV files of all reviews (git-tracked, for the human review)
|
||||
# --log-dir=DIR folder for the progress file (default: a temporary folder)
|
||||
# --resume-from=NAME continue after this checkpoint, e.g. --resume-from=taxonomy1b
|
||||
# --list only show the chunks and checkpoints, and stop
|
||||
|
||||
script_file <- "data-raw/_reproduction_scripts/reproduction_of_microorganisms.R"
|
||||
if (!file.exists(script_file)) {
|
||||
stop("Run this from the root of the AMR repository", call. = FALSE)
|
||||
}
|
||||
|
||||
# arguments
|
||||
args <- commandArgs(trailingOnly = TRUE)
|
||||
get_arg <- function(name, default = NULL) {
|
||||
hit <- args[startsWith(args, paste0("--", name, "="))]
|
||||
if (length(hit) == 0) {
|
||||
return(default)
|
||||
}
|
||||
sub("^[^=]+=", "", hit[1])
|
||||
}
|
||||
review_dir <- get_arg("review-dir")
|
||||
log_dir <- get_arg("log-dir", tempfile("microorganisms_build_"))
|
||||
resume_from <- get_arg("resume-from")
|
||||
dir.create(log_dir, recursive = TRUE, showWarnings = FALSE)
|
||||
progress_file <- file.path(log_dir, "progress.tsv")
|
||||
|
||||
# cut the script into chunks ----
|
||||
|
||||
lines <- readLines(script_file, encoding = "UTF-8")
|
||||
# the checkpoints in their order; taxonomy_lpsn_missing.rds is no checkpoint to resume from, as the rest of the
|
||||
# LPSN chunk is needed anyway (and is fast thanks to the LPSN cache)
|
||||
checkpoints <- c(
|
||||
"taxonomy_lpsn", "taxonomy_mycobank", "taxonomy_gbif",
|
||||
"taxonomy0", "taxonomy1", "taxonomy1b", "taxonomy1c",
|
||||
"taxonomy2", "taxonomy2b", "taxonomy2c", "taxonomy3", "taxonomy3b"
|
||||
)
|
||||
checkpoint_line <- vapply(checkpoints, function(cp) {
|
||||
hit <- which(startsWith(lines, "saveRDS(") & grepl(paste0("\"data-raw/", cp, ".rds\""), lines, fixed = TRUE))
|
||||
if (length(hit) != 1) {
|
||||
stop("Checkpoint ", cp, " must be saved exactly once in ", script_file, " (found ", length(hit), ")", call. = FALSE)
|
||||
}
|
||||
hit
|
||||
}, integer(1))
|
||||
if (is.unsorted(checkpoint_line)) {
|
||||
stop("The checkpoints in the script are not in the expected order", call. = FALSE)
|
||||
}
|
||||
setup_end <- which(startsWith(lines, "# Read LPSN data")) - 1
|
||||
stopifnot(length(setup_end) == 1)
|
||||
chunks <- data.frame(
|
||||
name = c("setup", checkpoints, "save_to_package"),
|
||||
from = c(1L, setup_end + 1L, checkpoint_line + 1L),
|
||||
to = c(setup_end, checkpoint_line, length(lines)),
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
|
||||
# the blocks that are needed again when resuming
|
||||
block_start <- which(startsWith(lines, "# @resume-block "))
|
||||
block_end <- which(lines == "# @end-resume-block")
|
||||
stopifnot(length(block_start) == length(block_end), all(block_start < block_end))
|
||||
blocks <- data.frame(
|
||||
name = sub("^# @resume-block ([^ ]+) .*", "\\1", lines[block_start]),
|
||||
after = sub(".* after=([^ ]+).*", "\\1", lines[block_start]),
|
||||
from = block_start,
|
||||
to = block_end,
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
stopifnot(all(blocks$after %in% checkpoints))
|
||||
|
||||
message("Chunks:")
|
||||
for (i in seq_len(nrow(chunks))) {
|
||||
message(sprintf(" %-17s lines %5d-%5d", chunks$name[i], chunks$from[i], chunks$to[i]))
|
||||
}
|
||||
message("Blocks needed when resuming:")
|
||||
for (i in seq_len(nrow(blocks))) {
|
||||
message(sprintf(" %-17s lines %5d-%5d, when resuming from %s or later", blocks$name[i], blocks$from[i], blocks$to[i], blocks$after[i]))
|
||||
}
|
||||
if ("--list" %in% args) {
|
||||
quit(save = "no", status = 0)
|
||||
}
|
||||
|
||||
# run ----
|
||||
|
||||
options(
|
||||
AMR_build_view = FALSE,
|
||||
AMR_build_print_rows = 200,
|
||||
warn = 1,
|
||||
width = 200
|
||||
)
|
||||
if (!is.null(review_dir)) {
|
||||
options(AMR_build_review_dir = review_dir)
|
||||
}
|
||||
if (!file.exists(progress_file)) {
|
||||
writeLines("chunk\tstarted\tfinished\tminutes\tstatus", progress_file)
|
||||
}
|
||||
log_progress <- function(chunk, started, status) {
|
||||
cat(
|
||||
paste(chunk, format(started, "%Y-%m-%d %H:%M:%S"), format(Sys.time(), "%Y-%m-%d %H:%M:%S"),
|
||||
round(as.numeric(difftime(Sys.time(), started, units = "mins")), 1), status,
|
||||
sep = "\t"
|
||||
), "\n",
|
||||
file = progress_file, append = TRUE, sep = ""
|
||||
)
|
||||
}
|
||||
run_lines <- function(name, from, to) {
|
||||
message("\n", strrep("=", 100), "\n=== CHUNK ", name, " (lines ", from, "-", to, ") started at ", format(Sys.time()), "\n", strrep("=", 100))
|
||||
started <- Sys.time()
|
||||
code <- paste(lines[from:to], collapse = "\n")
|
||||
ok <- tryCatch(
|
||||
{
|
||||
source(exprs = parse(text = code, keep.source = FALSE), local = globalenv(), echo = TRUE, max.deparse.length = 300)
|
||||
TRUE
|
||||
},
|
||||
error = function(e) {
|
||||
message("\n!!! CHUNK ", name, " FAILED: ", conditionMessage(e))
|
||||
log_progress(name, started, paste("FAILED:", gsub("[\t\n]", " ", conditionMessage(e))))
|
||||
FALSE
|
||||
}
|
||||
)
|
||||
if (!ok) {
|
||||
last_done <- which(chunks$name == name) - 2
|
||||
if (length(last_done) == 1 && last_done >= 1) {
|
||||
message("\nAfter fixing the cause, resume with: --resume-from=", checkpoints[last_done])
|
||||
} else {
|
||||
message("\nAfter fixing the cause, run again from the start")
|
||||
}
|
||||
quit(save = "no", status = 1)
|
||||
}
|
||||
log_progress(name, started, "OK")
|
||||
message("=== CHUNK ", name, " finished in ", round(as.numeric(difftime(Sys.time(), started, units = "mins")), 1), " minutes")
|
||||
}
|
||||
|
||||
run_lines("setup", chunks$from[1], chunks$to[1])
|
||||
|
||||
if (is.null(resume_from)) {
|
||||
first_chunk <- 2
|
||||
} else {
|
||||
if (!resume_from %in% checkpoints) {
|
||||
stop("--resume-from must be one of: ", paste(checkpoints, collapse = ", "), call. = FALSE)
|
||||
}
|
||||
k <- which(checkpoints == resume_from)
|
||||
message("\nResuming after checkpoint ", resume_from)
|
||||
started <- Sys.time()
|
||||
# the source taxonomies that were saved up to this checkpoint
|
||||
for (src in intersect(checkpoints[seq_len(k)], c("taxonomy_lpsn", "taxonomy_mycobank", "taxonomy_gbif"))) {
|
||||
assign(src, readRDS(file.path("data-raw", paste0(src, ".rds"))), envir = globalenv())
|
||||
}
|
||||
# the parts of earlier chunks that later chunks need
|
||||
for (i in which(match(blocks$after, checkpoints) <= k)) {
|
||||
run_lines(paste0("resume block ", blocks$name[i]), blocks$from[i], blocks$to[i])
|
||||
}
|
||||
if (!resume_from %in% c("taxonomy_lpsn", "taxonomy_mycobank", "taxonomy_gbif")) {
|
||||
assign("taxonomy", readRDS(file.path("data-raw", paste0(resume_from, ".rds"))), envir = globalenv())
|
||||
}
|
||||
log_progress(paste("resume from", resume_from), started, "OK")
|
||||
first_chunk <- which(chunks$name == resume_from) + 1
|
||||
}
|
||||
|
||||
for (i in seq(first_chunk, nrow(chunks))) {
|
||||
run_lines(chunks$name[i], chunks$from[i], chunks$to[i])
|
||||
}
|
||||
message("\nAll chunks finished. Progress file: ", progress_file)
|
||||
@@ -0,0 +1,39 @@
|
||||
# MO code registry
|
||||
|
||||
MO codes (such as `B_ESCHR_COLI`) are stored by users in their own data, sometimes for years. A code must
|
||||
therefore **always mean the same taxon**, in every future version of the AMR package. This folder contains
|
||||
the registry that guarantees this.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `mo_code_registry.csv` | Every MO code that was ever part of a **released** version of the AMR package since v2.0.0, with the taxon it denotes (name, rank, domain, genus, species, subspecies), the first and last release it was in, and older names of the same code (e.g. a corrected spelling). |
|
||||
| `build_mo_code_registry.R` | Recreates `mo_code_registry.csv` from the git tags of all releases. Run it after every new release. |
|
||||
| `mo_code_renames.csv` | The only allowed exceptions: registered codes that get another name, because it is the same taxon (e.g. a corrected spelling or a renamed species group). Every row needs `approved_by` and `approved_date`. |
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Only releases count.** Development versions (the main branch) are never part of the registry. Codes that
|
||||
only existed in a development version carry no weight and may be reassigned.
|
||||
2. **Codes from before v2.0.0 are not supported.** Decision by Matthijs S. Berends, 3 October 2026: MO codes
|
||||
from releases before v2.0.0 (12 March 2023) are not supported. Every code in all releases since v2.0.0 has
|
||||
always meant the same taxon (checked on 3 October 2026), so the registry starts without conflicts.
|
||||
3. **A registered code is never given to another taxon.** The taxonomy build
|
||||
(`data-raw/_reproduction_scripts/reproduction_of_microorganisms.R`) reuses the code of every taxon in the
|
||||
registry (most recent release first), never gives a registered code to a new taxon, and stops if a
|
||||
registered code would get another name than in the registry, unless that is listed and approved in
|
||||
`mo_code_renames.csv`.
|
||||
4. **A registered code that is no longer in the data still works.** `as.mo()` translates it to the current
|
||||
code of the same taxon (by its name, so also if the taxon moved to another domain), using an internal lookup
|
||||
table (`MO_RETIRED_CODES`) that is created from the registry in `data-raw/_pre_commit_checks.R`. If the
|
||||
taxon is no longer in the data at all, the result is `NA` with a warning. The package never guesses.
|
||||
5. **Tests enforce all of this.** `tests/testthat/test-data-microorganisms.R` checks the current data against
|
||||
the registry and the renames, and checks that every registered code resolves to its own taxon.
|
||||
|
||||
## After a release
|
||||
|
||||
1. Make sure the git tag of the release exists (e.g. `v3.1.0`).
|
||||
2. Run `source("data-raw/microorganisms_files/build_mo_code_registry.R")` from the root of the repository.
|
||||
3. Review the changes to `mo_code_registry.csv` (`git diff`): only new codes may be added, and existing codes
|
||||
may only get a later `last_release`. Commit the result.
|
||||
@@ -0,0 +1,113 @@
|
||||
# ==================================================================== #
|
||||
# TITLE: #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data #
|
||||
# #
|
||||
# SOURCE CODE: #
|
||||
# https://github.com/msberends/AMR #
|
||||
# #
|
||||
# PLEASE CITE THIS SOFTWARE AS: #
|
||||
# Berends MS, Luz CF, Friedrich AW, et al. (2022). #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data. #
|
||||
# Journal of Statistical Software, 104(3), 1-31. #
|
||||
# https://doi.org/10.18637/jss.v104.i03 #
|
||||
# #
|
||||
# Developed at the University of Groningen and the University Medical #
|
||||
# Center Groningen in the Netherlands, in collaboration with many #
|
||||
# colleagues from around the world, see our website. #
|
||||
# #
|
||||
# This R package is free software; you can freely use and distribute #
|
||||
# it for both personal and commercial purposes under the terms of the #
|
||||
# GNU General Public License version 2.0 (GNU GPL-2), as published by #
|
||||
# the Free Software Foundation. #
|
||||
# We created this package for both routine data analysis and academic #
|
||||
# research and it was publicly released in the hope that it will be #
|
||||
# useful, but it comes WITHOUT ANY WARRANTY OR LIABILITY. #
|
||||
# #
|
||||
# Visit our website for the full manual and a complete tutorial about #
|
||||
# how to conduct AMR data analysis: https://amr-for-r.org #
|
||||
# ==================================================================== #
|
||||
|
||||
# This script (re)creates data-raw/microorganisms_files/mo_code_registry.csv: the registry of all MO codes
|
||||
# that were ever part of a released version of the AMR package, since v2.0.0. See the README.md in this
|
||||
# folder for the rules.
|
||||
#
|
||||
# Run it from the root of the repository, after every new release (once its git tag exists):
|
||||
# source("data-raw/microorganisms_files/build_mo_code_registry.R")
|
||||
# It always rebuilds the whole registry from the git tags, so it is reproducible and can be run at any time.
|
||||
# Development versions (the main branch) are never part of the registry.
|
||||
|
||||
library(dplyr)
|
||||
devtools::load_all(".", quiet = TRUE) # for %like_case%
|
||||
|
||||
registry_file <- "data-raw/microorganisms_files/mo_code_registry.csv"
|
||||
|
||||
# Decision by Matthijs S. Berends, 3 October 2026: MO codes from releases before v2.0.0 (12 March 2023)
|
||||
# are not supported.
|
||||
first_supported_release <- "2.0.0"
|
||||
|
||||
# all release tags (vX.Y.Z, so no development versions) since the first supported release
|
||||
tags <- system2("git", "tag", stdout = TRUE)
|
||||
tags <- tags[tags %like_case% "^v[0-9]+[.][0-9]+[.][0-9]+$"]
|
||||
tags <- tags[numeric_version(sub("^v", "", tags)) >= numeric_version(first_supported_release)]
|
||||
tags <- tags[order(numeric_version(sub("^v", "", tags)))]
|
||||
message("Releases: ", toString(tags))
|
||||
|
||||
read_release <- function(tag) {
|
||||
tmp <- tempfile(fileext = ".rda")
|
||||
on.exit(unlink(tmp))
|
||||
status <- system2("git", c("show", paste0(tag, ":data/microorganisms.rda")), stdout = tmp)
|
||||
if (!identical(status, 0L)) {
|
||||
stop("Could not read data/microorganisms.rda of ", tag, call. = FALSE)
|
||||
}
|
||||
env <- new.env()
|
||||
load(tmp, envir = env)
|
||||
mo <- as.data.frame(env$microorganisms, stringsAsFactors = FALSE)
|
||||
# until v3.0.1, `kingdom` contained what is now called `domain`
|
||||
if (!"domain" %in% colnames(mo)) {
|
||||
mo$domain <- mo$kingdom
|
||||
mo$domain[mo$domain == "(unknown kingdom)"] <- "(unknown domain)"
|
||||
}
|
||||
data.frame(
|
||||
release = tag,
|
||||
mo = as.character(mo$mo),
|
||||
fullname = mo$fullname,
|
||||
rank = mo$rank,
|
||||
domain = mo$domain,
|
||||
genus = mo$genus,
|
||||
species = mo$species,
|
||||
subspecies = mo$subspecies,
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
}
|
||||
|
||||
all_releases <- bind_rows(lapply(tags, read_release)) %>%
|
||||
mutate(release_order = match(release, tags))
|
||||
|
||||
# one row per MO code; the name, rank and taxonomy are taken from the most recent release that contained the code
|
||||
registry <- all_releases %>%
|
||||
group_by(mo) %>%
|
||||
arrange(desc(release_order), .by_group = TRUE) %>%
|
||||
summarise(
|
||||
# other names the code had in older releases (e.g. a corrected subspecies name), for transparency
|
||||
# (must come first, since `fullname` is overwritten below)
|
||||
previous_names = paste(setdiff(unique(fullname), first(fullname)), collapse = "; "),
|
||||
fullname = first(fullname),
|
||||
rank = first(rank),
|
||||
domain = first(domain),
|
||||
genus = first(genus),
|
||||
species = first(species),
|
||||
subspecies = first(subspecies),
|
||||
first_release = tags[min(release_order)],
|
||||
last_release = tags[max(release_order)],
|
||||
.groups = "drop"
|
||||
) %>%
|
||||
relocate(previous_names, .after = everything()) %>%
|
||||
arrange(mo)
|
||||
|
||||
message(
|
||||
nrow(registry), " MO codes, of which ", sum(registry$previous_names != ""),
|
||||
" had another name in an older release (check these when the registry changes)"
|
||||
)
|
||||
|
||||
write.csv(registry, registry_file, row.names = FALSE, na = "")
|
||||
message("Saved to ", registry_file)
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
"mo","registry_name","new_name","reason","approved_by","approved_date"
|
||||
"B_MYCBC_AVIM-C","Mycobacterium avium-intracellulare complex","Mycobacterium avium complex","Species group renamed in the development version of 2026; same group of organisms","",""
|
||||
|
@@ -0,0 +1,225 @@
|
||||
# ==================================================================== #
|
||||
# TITLE: #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data #
|
||||
# #
|
||||
# SOURCE CODE: #
|
||||
# https://github.com/msberends/AMR #
|
||||
# #
|
||||
# PLEASE CITE THIS SOFTWARE AS: #
|
||||
# Berends MS, Luz CF, Friedrich AW, et al. (2022). #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data. #
|
||||
# Journal of Statistical Software, 104(3), 1-31. #
|
||||
# https://doi.org/10.18637/jss.v104.i03 #
|
||||
# #
|
||||
# Developed at the University of Groningen and the University Medical #
|
||||
# Center Groningen in the Netherlands, in collaboration with many #
|
||||
# colleagues from around the world, see our website. #
|
||||
# #
|
||||
# This R package is free software; you can freely use and distribute #
|
||||
# it for both personal and commercial purposes under the terms of the #
|
||||
# GNU General Public License version 2.0 (GNU GPL-2), as published by #
|
||||
# the Free Software Foundation. #
|
||||
# We created this package for both routine data analysis and academic #
|
||||
# research and it was publicly released in the hope that it will be #
|
||||
# useful, but it comes WITHOUT ANY WARRANTY OR LIABILITY. #
|
||||
# #
|
||||
# Visit our website for the full manual and a complete tutorial about #
|
||||
# how to conduct AMR data analysis: https://amr-for-r.org #
|
||||
# ==================================================================== #
|
||||
|
||||
# Integrity rules of the `microorganisms` data set.
|
||||
# This file is used by the unit tests (test-data-microorganisms.R) AND by the taxonomy build
|
||||
# (data-raw/_reproduction_scripts/reproduction_of_microorganisms.R), which stops before saving if any rule
|
||||
# is broken. So the rules exist in one place only.
|
||||
|
||||
# the MO code prefix of each domain
|
||||
MO_DOMAIN_PREFIX <- c(
|
||||
"Animalia" = "AN",
|
||||
"Archaea" = "A",
|
||||
"Bacteria" = "B",
|
||||
"Chromista" = "C",
|
||||
"Fungi" = "F",
|
||||
"Plantae" = "PL",
|
||||
"Protozoa" = "P"
|
||||
)
|
||||
|
||||
# read the MO code registry and the approved renames (see data-raw/microorganisms_files/README.md),
|
||||
# returns NULL if the files are not available (such as in an installed package)
|
||||
read_mo_registry <- function(root = mo_repository_root()) {
|
||||
file <- file.path(root, "data-raw", "microorganisms_files", "mo_code_registry.csv")
|
||||
if (is.null(root) || !file.exists(file)) {
|
||||
return(NULL)
|
||||
}
|
||||
utils::read.csv(file, colClasses = "character", na.strings = character(0))
|
||||
}
|
||||
read_mo_renames <- function(root = mo_repository_root()) {
|
||||
file <- file.path(root, "data-raw", "microorganisms_files", "mo_code_renames.csv")
|
||||
if (is.null(root) || !file.exists(file)) {
|
||||
return(NULL)
|
||||
}
|
||||
renames <- utils::read.csv(file, colClasses = "character", na.strings = character(0))
|
||||
# only approved renames count
|
||||
renames[renames$approved_by != "" & renames$approved_date != "", , drop = FALSE]
|
||||
}
|
||||
mo_repository_root <- function() {
|
||||
# works from the root of the repository and from tests/testthat
|
||||
for (path in c(".", "../..", "..")) {
|
||||
if (file.exists(file.path(path, "data-raw", "microorganisms_files", "mo_code_registry.csv"))) {
|
||||
return(normalizePath(path))
|
||||
}
|
||||
}
|
||||
NULL
|
||||
}
|
||||
|
||||
# names without a rank suffix, e.g. "Kapabacteria {class}" and "Nitrospira (class)" are both the class
|
||||
mo_name_without_suffix <- function(x) {
|
||||
trimws(gsub(" [{(][a-z ]+[)}]$", "", x))
|
||||
}
|
||||
|
||||
# Returns a named list with one character vector per rule, each describing the records that break the rule.
|
||||
# All vectors must be empty.
|
||||
mo_integrity_issues <- function(df, registry = NULL, renames = NULL) {
|
||||
df <- as.data.frame(df, stringsAsFactors = FALSE)
|
||||
df$mo <- as.character(df$mo)
|
||||
describe <- function(rows) {
|
||||
if (!any(rows, na.rm = TRUE)) {
|
||||
return(character(0))
|
||||
}
|
||||
rows <- which(rows)
|
||||
paste0(df$mo[rows], " (", df$fullname[rows], ")")
|
||||
}
|
||||
issues <- list()
|
||||
|
||||
# identifiers
|
||||
issues$duplicated_mo <- describe(duplicated(df$mo) | duplicated(df$mo, fromLast = TRUE))
|
||||
issues$duplicated_fullname <- describe(duplicated(df$fullname) | duplicated(df$fullname, fromLast = TRUE))
|
||||
issues$empty_mo <- describe(df$mo %in% c("", NA))
|
||||
issues$invalid_mo_characters <- describe(df$mo %unlike_case% "^[][A-Z0-9_-]+$")
|
||||
issues$repeated_mo_elements <- describe(df$mo %like_case% "^([A-Z]+_[^_]+)_\\1_")
|
||||
|
||||
# the code prefix must match the domain
|
||||
prefix <- sub("_.*", "", df$mo)
|
||||
expected_prefix <- unname(MO_DOMAIN_PREFIX[df$domain])
|
||||
issues$prefix_not_matching_domain <- describe(
|
||||
df$mo != "UNKNOWN" & !is.na(expected_prefix) & prefix != expected_prefix
|
||||
)
|
||||
issues$unknown_domain <- describe(!df$domain %in% c(names(MO_DOMAIN_PREFIX), "(unknown domain)"))
|
||||
|
||||
# the code of a species must start with the code of its genus, of a subspecies with the code of its species
|
||||
is_taxon <- df$fullname %unlike% "unknown"
|
||||
genus_mo <- df$mo[df$rank == "genus"][match(
|
||||
paste(df$domain, df$genus),
|
||||
paste(df$domain[df$rank == "genus"], df$genus[df$rank == "genus"])
|
||||
)]
|
||||
species_mo <- df$mo[df$rank == "species"][match(
|
||||
paste(df$domain, df$genus, df$species),
|
||||
paste(df$domain[df$rank == "species"], df$genus[df$rank == "species"], df$species[df$rank == "species"])
|
||||
)]
|
||||
issues$species_without_genus <- describe(df$rank == "species" & is_taxon & is.na(genus_mo))
|
||||
issues$species_code_not_under_genus <- describe(
|
||||
df$rank == "species" & is_taxon & !is.na(genus_mo) & !startsWith(df$mo, paste0(genus_mo, "_"))
|
||||
)
|
||||
# (Salmonella serovars are coded as genus + serovar by design, e.g. B_SLMNL_ACHN for Salmonella Aachen)
|
||||
issues$subspecies_code_not_under_species <- describe(
|
||||
df$rank == "subspecies" & is_taxon & df$genus != "Salmonella" &
|
||||
!is.na(species_mo) & !startsWith(df$mo, paste0(species_mo, "_"))
|
||||
)
|
||||
issues$subspecies_code_not_under_genus <- describe(
|
||||
df$rank == "subspecies" & is_taxon & !is.na(genus_mo) & !startsWith(df$mo, paste0(genus_mo, "_"))
|
||||
)
|
||||
|
||||
# sources can only contain their own scope (issue #309)
|
||||
issues$lpsn_outside_prokaryotes <- describe(df$source == "LPSN" & !df$domain %in% c("Bacteria", "Archaea"))
|
||||
issues$mycobank_outside_fungi <- describe(df$source == "MycoBank" & df$domain != "Fungi")
|
||||
|
||||
# allowed values
|
||||
issues$invalid_status <- describe(!df$status %in% c("accepted", "synonym", "unknown"))
|
||||
issues$invalid_rank <- describe(!df$rank %in% c(
|
||||
"domain", "kingdom", "phylum", "class", "order", "family", "genus", "species", "subspecies",
|
||||
"species group", "(unknown rank)"
|
||||
))
|
||||
issues$invalid_source <- describe(!df$source %in% c("LPSN", "MycoBank", "GBIF", "manually added"))
|
||||
issues$invalid_prevalence <- describe(is.na(df$prevalence) | df$prevalence < 1 | df$prevalence > 2)
|
||||
|
||||
# synonyms with a current name must lead to an existing record, without cycles or endless chains; a chain
|
||||
# that ends at a synonym without a current name is incomplete data, see mo_synonyms_without_current_name()
|
||||
if (all(c("lpsn_renamed_to", "gbif_renamed_to") %in% colnames(df))) {
|
||||
chain <- mo_synonym_chain_end(df)
|
||||
issues$synonym_with_unknown_target <- describe(chain$has_target & is.na(chain$end))
|
||||
issues$synonym_chain_not_ending <- describe(
|
||||
chain$has_target & !is.na(chain$end) & chain$end_is_synonym & chain$end_has_target
|
||||
)
|
||||
}
|
||||
|
||||
# the MO code registry: a registered code must always denote the same taxon
|
||||
if (!is.null(registry)) {
|
||||
registered_name <- registry$fullname[match(df$mo, registry$mo)]
|
||||
if (!is.null(renames) && nrow(renames) > 0) {
|
||||
renamed <- match(df$mo, renames$mo)
|
||||
registered_name[!is.na(renamed)] <- renames$new_name[renamed[!is.na(renamed)]]
|
||||
}
|
||||
issues$registered_code_with_other_taxon <- describe(
|
||||
!is.na(registered_name) & mo_name_without_suffix(registered_name) != mo_name_without_suffix(df$fullname)
|
||||
)
|
||||
}
|
||||
|
||||
issues
|
||||
}
|
||||
|
||||
# the first available 'renamed to' identifier of each record
|
||||
mo_renamed_to <- function(df) {
|
||||
out <- df$lpsn_renamed_to
|
||||
if ("mycobank_renamed_to" %in% colnames(df)) {
|
||||
out <- ifelse(is.na(out), df$mycobank_renamed_to, out)
|
||||
}
|
||||
ifelse(is.na(out), df$gbif_renamed_to, out)
|
||||
}
|
||||
|
||||
# for every synonym with a target: where its chain of renames ends
|
||||
mo_synonym_chain_end <- function(df) {
|
||||
df <- as.data.frame(df, stringsAsFactors = FALSE)
|
||||
df$mo <- as.character(df$mo)
|
||||
has_target <- df$status == "synonym" & !is.na(mo_renamed_to(df))
|
||||
end <- rep(NA_character_, nrow(df))
|
||||
end[has_target] <- AMR:::synonym_mo_to_accepted_mo(df$mo[has_target], fill_in_accepted = FALSE, dataset = df)
|
||||
end_row <- match(end, df$mo)
|
||||
# a target only counts if it exists in the data set
|
||||
known_target <- (!is.na(df$lpsn_renamed_to) & df$lpsn_renamed_to %in% df$lpsn) |
|
||||
(!is.na(df$gbif_renamed_to) & df$gbif_renamed_to %in% df$gbif)
|
||||
if ("mycobank_renamed_to" %in% colnames(df)) {
|
||||
known_target <- known_target | (!is.na(df$mycobank_renamed_to) & df$mycobank_renamed_to %in% df$mycobank)
|
||||
}
|
||||
data.frame(
|
||||
has_target = has_target,
|
||||
end = end,
|
||||
end_is_synonym = df$status[end_row] %in% "synonym",
|
||||
end_has_target = known_target[end_row] %in% TRUE
|
||||
)
|
||||
}
|
||||
|
||||
# synonyms for which no current name is known (e.g. because the record of the current name was not included),
|
||||
# directly or at the end of their chain of renames (e.g. Candida glabratus -> Nakaseomyces glabrata, which has no
|
||||
# current name itself); these are no error, but should be as few as possible
|
||||
mo_synonyms_without_current_name <- function(df) {
|
||||
df <- as.data.frame(df, stringsAsFactors = FALSE)
|
||||
chain <- mo_synonym_chain_end(df)
|
||||
no_target <- df$status == "synonym" & !chain$has_target
|
||||
dead_end_chain <- chain$has_target & !is.na(chain$end) & chain$end_is_synonym & !chain$end_has_target
|
||||
out <- df[no_target | dead_end_chain, c("mo", "fullname", "source"), drop = FALSE]
|
||||
out$chain_ends_at <- df$fullname[match(chain$end[no_target | dead_end_chain], df$mo)]
|
||||
out
|
||||
}
|
||||
|
||||
# a readable summary of all broken rules, or NULL if there are none
|
||||
mo_integrity_report <- function(issues) {
|
||||
issues <- issues[lengths(issues) > 0]
|
||||
if (length(issues) == 0) {
|
||||
return(NULL)
|
||||
}
|
||||
paste0(
|
||||
names(issues), " (", lengths(issues), "): ",
|
||||
vapply(issues, function(x) paste(utils::head(x, 10), collapse = ", "), character(1)),
|
||||
ifelse(lengths(issues) > 10, ", ...", ""),
|
||||
collapse = "\n"
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,408 @@
|
||||
"rule","record"
|
||||
"duplicated_fullname","AN_ANSKS_THLI (Anisakis kukenthalii)"
|
||||
"duplicated_fullname","AN_ANSKS_NTHLI (Anisakis kukenthalii)"
|
||||
"duplicated_fullname","F_BPLRS_BCHL (Bipolaris buchloes)"
|
||||
"duplicated_fullname","F_BPLRS_CHLS (Bipolaris buchloes)"
|
||||
"duplicated_fullname","F_CLDSP_EPCH (Cladosporium epichloes)"
|
||||
"duplicated_fullname","F_CLDSP_EPCHL (Cladosporium epichloes)"
|
||||
"duplicated_fullname","F_FUSRM_ALOS (Fusarium aloes)"
|
||||
"duplicated_fullname","F_FUSRM_LO""ES (Fusarium aloes)"
|
||||
"duplicated_fullname","B_GLCPS_NGNA (Gloeocapsa kutzingiana)"
|
||||
"duplicated_fullname","B_GLCPS_ZNGNA (Gloeocapsa kutzingiana)"
|
||||
"duplicated_fullname","B_HTRLB_ZNGI (Heteroleibleinia kutzingii)"
|
||||
"duplicated_fullname","B_HTRLB_TZNGI (Heteroleibleinia kutzingii)"
|
||||
"duplicated_fullname","F_NECTR_EPCH (Nectria epichloe)"
|
||||
"duplicated_fullname","F_NECTR_CHL""E (Nectria epichloe)"
|
||||
"duplicated_fullname","P_PRTSP_DYBS (Protospongia dybsoeensis)"
|
||||
"duplicated_fullname","P_PRTSP_SNSS (Protospongia dybsoeensis)"
|
||||
"duplicated_fullname","F_SYNCP_ALMA (Syncephalastrum alma-ataense)"
|
||||
"duplicated_fullname","F_SYNCP_ANSE (Syncephalastrum alma-ataense)"
|
||||
"duplicated_fullname","F_USTLG_BCHL (Ustilago buchloeformis)"
|
||||
"duplicated_fullname","F_USTLG_BUCHLS (Ustilago buchloes)"
|
||||
"duplicated_fullname","F_USTLG_BCHLF (Ustilago buchloeformis)"
|
||||
"duplicated_fullname","F_USTLG_BCHL""ES (Ustilago buchloes)"
|
||||
"duplicated_fullname","F_USTLG_ISTS (Ustilago isoetis)"
|
||||
"duplicated_fullname","F_USTLG_STIS (Ustilago isoetis)"
|
||||
"invalid_mo_characters","F_FUSRM_LO""ES (Fusarium aloes)"
|
||||
"invalid_mo_characters","F_NECTR_CHL""E (Nectria epichloe)"
|
||||
"invalid_mo_characters","F_NECTR_CHL""E_EPCH (Nectria epichloe epichloe)"
|
||||
"invalid_mo_characters","F_NECTR_CHL""E_ROSE (Nectria epichloe rosea)"
|
||||
"invalid_mo_characters","F_USTLG_BCHL""ES (Ustilago buchloes)"
|
||||
"subspecies_code_not_under_species","F_NECTR_CHL""E_EPCH (Nectria epichloe epichloe)"
|
||||
"subspecies_code_not_under_species","F_NECTR_CHL""E_ROSE (Nectria epichloe rosea)"
|
||||
"mycobank_outside_fungi","P_[FAM]_CULCSPRD (Culicosporidae)"
|
||||
"mycobank_outside_fungi","P_[FAM]_GOLBERGD (Golbergiidae)"
|
||||
"mycobank_outside_fungi","P_[FAM]_JANACEKD (Janacekiidae)"
|
||||
"mycobank_outside_fungi","B_PIRLL_CRCN (Pirella circinans)"
|
||||
"mycobank_outside_fungi","B_TRCHP (Trichophyton)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SNCM (Trichophyton abissinicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTM (Trichophyton acuminatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ACTL (Trichophyton acutulum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_AFRC (Trichophyton africanum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_AJLL (Trichophyton ajelloi)"
|
||||
"mycobank_outside_fungi","B_TRCHP_AJLL_AJLL (Trichophyton ajelloi ajelloi)"
|
||||
"mycobank_outside_fungi","B_TRCHP_AJLL_NANM (Trichophyton ajelloi nanum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ALBC (Trichophyton albiciscans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ALBM (Trichophyton album)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ANGL (Trichophyton angolense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ANNL (Trichophyton annulosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_APTH (Trichophyton apathogenicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ARLT (Trichophyton areolatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ASTR (Trichophyton asteroides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_AVRC (Trichophyton avaricum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_AVIM (Trichophyton avium)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LCNM (Trichophyton balcaneum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BTNR (Trichophyton batonrougei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BNHM (Trichophyton benhamiae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BNHM_BNHM (Trichophyton benhamiae benhamiae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BNHM_LUTM (Trichophyton benhamiae luteum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BCLR (Trichophyton bicolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BLNC (Trichophyton blanchardii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BRMP (Trichophyton brumptii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_BLLS (Trichophyton bullosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CBLL (Trichophyton caballinum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CMRN (Trichophyton camerounense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CNDL (Trichophyton candelabrum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CNNM (Trichophyton caninum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRTM (Trichophyton carateum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CSTN (Trichophyton castaneum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CSTL (Trichophyton castellanii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRBR (Trichophyton cerebriforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRTN (Trichophyton ceroton)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CYLN (Trichophyton ceylonense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CHBN (Trichophyton chibaense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CHSN (Trichophyton chosenicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CNRC (Trichophyton cineraceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CNRM (Trichophyton cinereum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRCN (Trichophyton circonvolutum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CCCN (Trichophyton coccineum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TRCM (Trichophyton concentricum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CNCM (Trichophyton conicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRNT (Trichophyton coronatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRTR (Trichophyton crateriforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRTR_CRTR (Trichophyton crateriforme crateriforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRTR_OCHR (Trichophyton crateriforme ochropyrraceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRTRF_CRTR (Trichophyton crateriformis crateriformis)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CRRS (Trichophyton cruris)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CURR (Trichophyton currii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DCTL (Trichophyton dactilistegicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DNKL (Trichophyton dankaliense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DCLV (Trichophyton decalvans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CLTM (Trichophyton denticulatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DPLN (Trichophyton depilans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DPRS (Trichophyton depressum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DRNN (Trichophyton dernense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_DSCD (Trichophyton discoides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EBRM (Trichophyton eboreum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RCTM (Trichophyton effractum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EHMN (Trichophyton ehimeense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EPLN (Trichophyton epilans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EQNM (Trichophyton equinum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EQNM_ATTR (Trichophyton equinum autotrophicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EQNM_EQNM (Trichophyton equinum equinum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ERNC (Trichophyton erinacei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ERTN (Trichophyton erioton)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ERTR (Trichophyton eriotrephon)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ERPM (Trichophyton europaeum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EVLC (Trichophyton evolceanui)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EXPN (Trichophyton expansum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_EXSC (Trichophyton exsiccatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LNTM (Trichophyton farinulentum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FVFR (Trichophyton faviforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FVFR_ALBM (Trichophyton faviforme album)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FVFR_DSCD (Trichophyton faviforme discoides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FVFR_FVFR (Trichophyton faviforme faviforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FVFR_OCHR (Trichophyton faviforme ochraceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLNM (Trichophyton felineum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FRRG (Trichophyton ferrugineum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FRRG_ALBM (Trichophyton ferrugineum album)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FRRG_FRRG (Trichophyton ferrugineum ferrugineum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FRRG_URGY (Trichophyton ferrugineum uruguayense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FSCH (Trichophyton fischeri)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLVS (Trichophyton flavescens)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLVV (Trichophyton flavovirens)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLVM (Trichophyton flavum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLRL (Trichophyton floreale)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLRF (Trichophyton floriforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_NNSE (Trichophyton fluviomuniense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FRMSN (Trichophyton formosanum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FLGN (Trichophyton fuligineum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FMTM (Trichophyton fumatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_FSCS (Trichophyton fuscosulcatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GLLN (Trichophyton gallinae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GLLP (Trichophyton gallopavum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GMLL (Trichophyton gamelleirae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GERG (Trichophyton georgiae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LBRM (Trichophyton glabrum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LBRM_FSCN (Trichophyton glabrum fuscinum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LBRM_GLBR (Trichophyton glabrum glabrum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GLOR (Trichophyton gloriae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GRVL (Trichophyton gourvilii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GRVL_GRVL (Trichophyton gourvilii gourvilii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_NLSM (Trichophyton granulosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GRSM (Trichophyton griseum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GZZN (Trichophyton guzzonii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS (Trichophyton gypseum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_ASTR (Trichophyton gypseum asteroides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_CHBN (Trichophyton gypseum chibaense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_GMML (Trichophyton gypseum gemmulolaniferum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_GRNL (Trichophyton gypseum granulosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_GYPS (Trichophyton gypseum gypseum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_HLCM (Trichophyton gypseum hiulcum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_KFMW (Trichophyton gypseum kaufmann-wolfii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_LCTC (Trichophyton gypseum lacticolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_QNCK (Trichophyton gypseum quinckeanum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_RDNS (Trichophyton gypseum radians)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_RDLT (Trichophyton gypseum radiolatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_RDPL (Trichophyton gypseum radioplicatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GYPS_SBFS (Trichophyton gypseum subfuscum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_IMMR (Trichophyton immergens)"
|
||||
"mycobank_outside_fungi","B_TRCHP_NDCM (Trichophyton indicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_INDT (Trichophyton indotineae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_INGN (Trichophyton inguinale)"
|
||||
"mycobank_outside_fungi","B_TRCHP_INTR (Trichophyton interdigitale)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RGNS (Trichophyton intertriginis)"
|
||||
"mycobank_outside_fungi","B_TRCHP_JPNC (Trichophyton japonicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_KGWN (Trichophyton kagawaense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_KANE (Trichophyton kanei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_KFMW (Trichophyton kaufmann-wolfii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_KRJD (Trichophyton krajdenii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_KRYN (Trichophyton kuryangei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LCTC (Trichophyton lacticolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LNGR (Trichophyton langeronii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LPFR (Trichophyton lipoferum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LNGF (Trichophyton longifusum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LSNC (Trichophyton louisianicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_LXRN (Trichophyton luxurians)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNSN (Trichophyton mansonii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MRGN (Trichophyton marginatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MART (Trichophyton mariatii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MGLS_ENDT (Trichophyton megalosporum endothrix)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MGLS_MGLS (Trichophyton megalosporum megalosporum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MGNN (Trichophyton megninii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG (Trichophyton mentagrophytes)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_ASTR (Trichophyton mentagrophytes asteroides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_BTNR (Trichophyton mentagrophytes batonrougei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_CCRB (Trichophyton mentagrophytes cucurbeticum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_ERNC (Trichophyton mentagrophytes erinacei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_GOTZ (Trichophyton mentagrophytes goetzii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_GRNL (Trichophyton mentagrophytes granulosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_GYPS (Trichophyton mentagrophytes gypseum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_INDT (Trichophyton mentagrophytes indotineae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_INTR (Trichophyton mentagrophytes interdigitale)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_LCTC (Trichophyton mentagrophytes lacticolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_MNTG (Trichophyton mentagrophytes mentagrophytes)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_NDLR (Trichophyton mentagrophytes nodulare)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_PRSC (Trichophyton mentagrophytes persicolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNTG_QNCK (Trichophyton mentagrophytes quinckeanum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MCRS (Trichophyton microsporum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MLCH (Trichophyton milochevitchii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MNMM (Trichophyton minimum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_MLTC (Trichophyton multicolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_NDFR (Trichophyton nodiformans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_OCNC (Trichophyton oceanicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_OCHT (Trichophyton ochoterenai)"
|
||||
"mycobank_outside_fungi","B_TRCHP_HRCM (Trichophyton ochraceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RRCM (Trichophyton ochropyrraceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_OLEX (Trichophyton olexae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ONYC (Trichophyton onychocola)"
|
||||
"mycobank_outside_fungi","B_TRCHP_OTAE (Trichophyton otae)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PPLL (Trichophyton papillosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PEDS (Trichophyton pedis)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PRSC (Trichophyton persicolor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PRSCM (Trichophyton persicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PRVS (Trichophyton pervesii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PHSL (Trichophyton phaseoliforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PCTR (Trichophyton pictor)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PLSM (Trichophyton pilosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PLCT (Trichophyton plicatile)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PLRZ (Trichophyton plurizoniforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PLYG (Trichophyton polygorum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PRTN (Trichophyton pratense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PRLF (Trichophyton proliferans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PRNS (Trichophyton pruinosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PSDT (Trichophyton pseudotonsurans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PTRY_PTRY (Trichophyton pterygoides pterygoides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_PTRY_VLCM (Trichophyton pterygoides violaceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RPRM (Trichophyton purpureum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_CKNM (Trichophyton quinckeanum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RDNS (Trichophyton radians)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RDCS (Trichophyton radicosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RDLT (Trichophyton radiolatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RDPL (Trichophyton radioplicatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBTS (Trichophyton raubitschekii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RDLL (Trichophyton redellii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RGLR (Trichophyton regulare)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RQTT (Trichophyton roquettei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ROSCM (Trichophyton rosaceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_ROSM (Trichophyton roseum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RTND (Trichophyton rotundum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBDM (Trichophyton rubidum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBRM (Trichophyton rubrum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBRM_FLVM (Trichophyton rubrum flavum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBRM_NGRC (Trichophyton rubrum nigricans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBRM_OCNC (Trichophyton rubrum oceanicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBRM_RDHN (Trichophyton rubrum rodhainii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_RBRM_RBRM (Trichophyton rubrum rubrum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SBRD (Trichophyton sabouraudii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SBRD_PLSM (Trichophyton sabouraudii pilosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SBRD_SBRD (Trichophyton sabouraudii sabouraudii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_STNS (Trichophyton saitense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SLTN (Trichophyton saltense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SRKS (Trichophyton sarkisovii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SCHN (Trichophyton schoenleinii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SCHN_GLCM (Trichophyton schoenleinii mongolicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SCHN_SCHN (Trichophyton schoenleinii schoenleinii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SERS (Trichophyton serisei)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SIMI (Trichophyton simii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SNGL (Trichophyton singulare)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SDNN (Trichophyton soudanense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SPDX (Trichophyton spadix)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SPRL (Trichophyton spiraliforme)"
|
||||
"mycobank_outside_fungi","B_TRCHP_SLFR (Trichophyton sulfureum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_THRN (Trichophyton teheraniense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSH (Trichophyton tenuishyphum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TRRS (Trichophyton terrestre)"
|
||||
"mycobank_outside_fungi","B_TRCHP_GNSE (Trichophyton thuringiense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR (Trichophyton tonsurans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_ACMN (Trichophyton tonsurans acuminatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_CLST (Trichophyton tonsurans closterosporum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_EFFR (Trichophyton tonsurans effractum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_EXSC (Trichophyton tonsurans exsiccatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_PRFR (Trichophyton tonsurans perforans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_SLFR (Trichophyton tonsurans sulfureum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_TNSR_TNSR (Trichophyton tonsurans tonsurans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_UMBL (Trichophyton umbilicatum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VNBR (Trichophyton vanbreuseghemii)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VRRC (Trichophyton verrucosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VRRC_ALBM (Trichophyton verrucosum album)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VRRC_ATTR (Trichophyton verrucosum autotrophicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VRRC_DSCD (Trichophyton verrucosum discoides)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VRRC_OCHR (Trichophyton verrucosum ochraceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VRRC_VRRC (Trichophyton verrucosum verrucosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VINN (Trichophyton viannai)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VLLS (Trichophyton villosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VNSM (Trichophyton vinosum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VLCM (Trichophyton violaceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VLCM_DCLV (Trichophyton violaceum decalvans)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VLCM_INDC (Trichophyton violaceum indicum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VLCM_KHRT (Trichophyton violaceum khartoumense)"
|
||||
"mycobank_outside_fungi","B_TRCHP_VLCM_VLCM (Trichophyton violaceum violaceum)"
|
||||
"mycobank_outside_fungi","B_TRCHP_YAND (Trichophyton yaoundei)"
|
||||
"mycobank_outside_fungi","B_TBRCL_GLND (Tubercularia glandicola)"
|
||||
"mycobank_outside_fungi","C_TBRCN_PNCT (Tuburcinia punctiformis)"
|
||||
"synonym_with_unknown_target","F_CHTMM_AMBR (Chaetomium amberpetense)"
|
||||
"synonym_with_unknown_target","F_CHTMM_FSSPR (Chaetomium fusisporale)"
|
||||
"synonym_with_unknown_target","F_CHTMM_PSDR (Chaetomium pseudoerraticum)"
|
||||
"synonym_with_unknown_target","F_CHTMM_RCTP (Chaetomium rectopilium)"
|
||||
"synonym_with_unknown_target","F_CHTMM_SRPN (Chaetomium serpentinum)"
|
||||
"synonym_with_unknown_target","F_CHTMM_TRTC (Chaetomium triticicola)"
|
||||
"synonym_with_unknown_target","F_CHTMM_VRRC (Chaetomium verrucichaeta)"
|
||||
"synonym_with_unknown_target","F_FUSRM_CEST (Fusarium cesatii)"
|
||||
"synonym_with_unknown_target","F_HNSNL (Hansenula)"
|
||||
"synonym_with_unknown_target","F_HNDRS_TGNA (Hendersonula fructigena)"
|
||||
"synonym_with_unknown_target","F_HNDRS_SYMP (Hendersonula symploci)"
|
||||
"synonym_with_unknown_target","F_HNDRS_TREV (Hendersonula trevoae)"
|
||||
"synonym_with_unknown_target","F_HPHLA (Hylophila)"
|
||||
"synonym_with_unknown_target","B_LPTBS_CCSC (Leptobasis caucasica)"
|
||||
"synonym_with_unknown_target","B_LPTBS_SHNX (Leptobasis shaanxiensis)"
|
||||
"synonym_with_unknown_target","F_[FAM]_LETROUTC (Letrouitiaceae)"
|
||||
"synonym_with_unknown_target","F_MTSCH_UNCS (Metschnikowia unicuspidata)"
|
||||
"synonym_with_unknown_target","B_MCRCH_CRSS (Microchaete crassa)"
|
||||
"synonym_with_unknown_target","B_MCRCH_STRT (Microchaete striatula)"
|
||||
"synonym_with_unknown_target","B_MCRCH_TNSS (Microchaete tenuissima)"
|
||||
"synonym_with_unknown_target","F_MRTHR (Morthiera)"
|
||||
"synonym_with_unknown_target","F_NAUMV (Naumovia)"
|
||||
"synonym_with_unknown_target","F_OMPHLR (Omphalaria)"
|
||||
"synonym_with_unknown_target","F_[FAM]_ORBILIAC (Orbiliaceae)"
|
||||
"synonym_with_unknown_target","P_PMPHG (Pamphagus)"
|
||||
"synonym_with_unknown_target","F_[FAM]_PELTULAC (Peltulaceae)"
|
||||
"synonym_with_unknown_target","F_PHOMA_FSCT (Phoma fuscata)"
|
||||
"synonym_with_unknown_target","F_PHMPS_CYDN (Phomopsis cydoniae)"
|
||||
"synonym_with_unknown_target","F_PLYSC (Polysaccum)"
|
||||
"synonym_with_unknown_target","P_RHMNS (Rhynchomonas)"
|
||||
"synonym_with_unknown_target","F_[FAM]_SAREACEA (Sareaceae)"
|
||||
"synonym_with_unknown_target","F_[FAM]_SCLRCCCC (Sclerococcaceae)"
|
||||
"synonym_with_unknown_target","F_SPRTR_BRNN (Sporotrichum brunneum)"
|
||||
"synonym_with_unknown_target","F_SPRTR_CLCG (Sporotrichum calcigenum)"
|
||||
"synonym_with_unknown_target","F_SPRTR_CNNB (Sporotrichum cinnabarinum)"
|
||||
"synonym_with_unknown_target","F_SPRTR_ELCH (Sporotrichum elaeochroum)"
|
||||
"synonym_with_unknown_target","F_SPRTR_HMSM (Sporotrichum hamosum)"
|
||||
"synonym_with_unknown_target","F_SPRTR_HLSR (Sporotrichum holosericeum)"
|
||||
"synonym_with_unknown_target","F_SPRTR_PRTN (Sporotrichum parietinum)"
|
||||
"synonym_with_unknown_target","F_USTLG_TULP (Ustilago tulipae)"
|
||||
"synonym_chain_not_ending","F_CPLLD_DNSPR (Capillidium denaeosporus)"
|
||||
"synonym_chain_not_ending","F_CPLLD_LBTS (Capillidium lobatus)"
|
||||
"synonym_chain_not_ending","F_CPLLD_RHYSS (Capillidium rhysosporus)"
|
||||
"synonym_chain_not_ending","F_CNDBL_DNSPR (Conidiobolus denaeosporum)"
|
||||
"synonym_chain_not_ending","F_CNDBL_LBTM (Conidiobolus lobatum)"
|
||||
"synonym_chain_not_ending","F_CNDBL_RHYSS (Conidiobolus rhysosporum)"
|
||||
"synonym_chain_not_ending","F_OCHRC_QTCM (Ochroconis aquaticum)"
|
||||
"synonym_chain_not_ending","F_OCHRC_LVCM (Ochroconis olivaceum)"
|
||||
"synonym_chain_not_ending","F_OCHRC_PHRM (Ochroconis phaeophorum)"
|
||||
"synonym_chain_not_ending","F_OCHRC_RMSM (Ochroconis ramosum)"
|
||||
"synonym_chain_not_ending","F_OCHRC_BSTM (Ochroconis robustum)"
|
||||
"synonym_chain_not_ending","F_OCHRC_RCSM (Ochroconis verrucosum)"
|
||||
"synonym_chain_not_ending","F_PGMDA_TRCM (Pseudosigmoidea excentricum)"
|
||||
"synonym_chain_not_ending","F_SCLCB_AQTC (Scolecobasidium aquatica)"
|
||||
"synonym_chain_not_ending","F_SCLCB_EXCNT (Scolecobasidium excentrica)"
|
||||
"synonym_chain_not_ending","F_SCLCB_OLVC (Scolecobasidium olivacea)"
|
||||
"synonym_chain_not_ending","F_SCLCB_PHPH (Scolecobasidium phaeophora)"
|
||||
"synonym_chain_not_ending","F_SCLCB_RAMS (Scolecobasidium ramosa)"
|
||||
"synonym_chain_not_ending","F_SCLCB_RBST (Scolecobasidium robusta)"
|
||||
"synonym_chain_not_ending","F_SCLCB_RCSA (Scolecobasidium verrucosa)"
|
||||
"registered_code_with_other_taxon","B_ALTBC (Alitibacter)"
|
||||
"registered_code_with_other_taxon","F_AMRPH (Amorphotheca)"
|
||||
"registered_code_with_other_taxon","F_AMPHC (Amphichorda)"
|
||||
"registered_code_with_other_taxon","F_ANTHR (Antherospora)"
|
||||
"registered_code_with_other_taxon","F_ARTHR (Arthroascus)"
|
||||
"registered_code_with_other_taxon","F_ASCPH_GRNL (Ascophanus granulatus)"
|
||||
"registered_code_with_other_taxon","F_ASTRM (Asteroma)"
|
||||
"registered_code_with_other_taxon","F_CHLMY (Chlamydoabsidia)"
|
||||
"registered_code_with_other_taxon","P_CHYTR (Chytridioides)"
|
||||
"registered_code_with_other_taxon","P_CHYTR_SCHZ (Chytridioides schizophylli)"
|
||||
"registered_code_with_other_taxon","F_CLVLN (Clavulina)"
|
||||
"registered_code_with_other_taxon","F_COLLM_CHLM (Collema cheilium)"
|
||||
"registered_code_with_other_taxon","F_CRTNR_CRCC (Cortinarius croceocoeruleus)"
|
||||
"registered_code_with_other_taxon","F_CRTNR_HRCN (Cortinarius hircinus)"
|
||||
"registered_code_with_other_taxon","F_CYLND (Cylindricascospora)"
|
||||
"registered_code_with_other_taxon","F_DMRSP (Dimerospora)"
|
||||
"registered_code_with_other_taxon","F_ERYSP_UMBL (Erysiphe umbilici)"
|
||||
"registered_code_with_other_taxon","B_FCLBCT_INTS (Faecalibacterium intestinale)"
|
||||
"registered_code_with_other_taxon","B_[FAM]_FRVDBCTR (Fervidibacteraceae)"
|
||||
"registered_code_with_other_taxon","B_[FAM]_FUSBCTRC (Fusibacteraceae)"
|
||||
"registered_code_with_other_taxon","B_HTRSC (Heteroscytonema)"
|
||||
"registered_code_with_other_taxon","B_HMSPH (Humisphingomonas)"
|
||||
"registered_code_with_other_taxon","F_HYDRP (Hydrophora)"
|
||||
"registered_code_with_other_taxon","F_IDRLL (Idriella)"
|
||||
"registered_code_with_other_taxon","F_LCNLL (Lecaniella)"
|
||||
"registered_code_with_other_taxon","F_LECID_GLMR (Lecidea glomerella)"
|
||||
"registered_code_with_other_taxon","F_LECID_SBLT (Lecidea subuletorum)"
|
||||
"registered_code_with_other_taxon","B_MYCBC_AVIM-C (Mycobacterium avium complex)"
|
||||
"registered_code_with_other_taxon","B_MYCBC_NNSE (Mycobacterium holstebronense)"
|
||||
"registered_code_with_other_taxon","F_NMSPR (Naemaspora)"
|
||||
"registered_code_with_other_taxon","F_NCNTH (Neoconiothyrium)"
|
||||
"registered_code_with_other_taxon","F_PRDPS (Parodiopsis)"
|
||||
"registered_code_with_other_taxon","B_PHYTP (Phytopseudomonas)"
|
||||
"registered_code_with_other_taxon","F_PLSPR_SCRP (Pleospora scirpicola)"
|
||||
"registered_code_with_other_taxon","F_PLYSC (Polysaccum)"
|
||||
"registered_code_with_other_taxon","F_PSDSP (Pseudoasperisporium)"
|
||||
"registered_code_with_other_taxon","B_PSDLC (Pseudolactococcus)"
|
||||
"registered_code_with_other_taxon","F_PSDPN (Pseudopenicillium)"
|
||||
"registered_code_with_other_taxon","F_PSRTC (Psorotichia)"
|
||||
"registered_code_with_other_taxon","F_RINDN_CNTR (Rinodina contribuens)"
|
||||
"registered_code_with_other_taxon","F_SPRSR_ANTH (Sporisorium anthephorae)"
|
||||
"registered_code_with_other_taxon","B_SMYCS_GLVS (Streptomyces gilvus)"
|
||||
"registered_code_with_other_taxon","F_ZYGRH (Zygorhynchus)"
|
||||
"sentinel","Tropheryma whipplei is missing"
|
||||
"sentinel","Ascaris lumbricoides is missing"
|
||||
"sentinel","Echinococcus granulosus is missing"
|
||||
"sentinel","Leishmania donovani is missing"
|
||||
"sentinel","Trichophyton rubrum domain is Bacteria instead of Fungi"
|
||||
"sentinel","Necator americanus domain is Fungi instead of Animalia"
|
||||
"sentinel","Capillaria domain is Fungi instead of Animalia"
|
||||
"sentinel","Trichophyton rubrum family is Oscillatoriaceae instead of Arthrodermataceae"
|
||||
"sentinel","Mucor family is instead of Mucoraceae"
|
||||
"sentinel","Trichophyton rubrum has a code with the wrong prefix"
|
||||
"sentinel","Necator americanus has a code with the wrong prefix"
|
||||
"sentinel","Capillaria has a code with the wrong prefix"
|
||||
|
@@ -0,0 +1,197 @@
|
||||
# ==================================================================== #
|
||||
# TITLE: #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data #
|
||||
# #
|
||||
# SOURCE CODE: #
|
||||
# https://github.com/msberends/AMR #
|
||||
# #
|
||||
# PLEASE CITE THIS SOFTWARE AS: #
|
||||
# Berends MS, Luz CF, Friedrich AW, et al. (2022). #
|
||||
# AMR: An R Package for Working with Antimicrobial Resistance Data. #
|
||||
# Journal of Statistical Software, 104(3), 1-31. #
|
||||
# https://doi.org/10.18637/jss.v104.i03 #
|
||||
# #
|
||||
# Developed at the University of Groningen and the University Medical #
|
||||
# Center Groningen in the Netherlands, in collaboration with many #
|
||||
# colleagues from around the world, see our website. #
|
||||
# #
|
||||
# This R package is free software; you can freely use and distribute #
|
||||
# it for both personal and commercial purposes under the terms of the #
|
||||
# GNU General Public License version 2.0 (GNU GPL-2), as published by #
|
||||
# the Free Software Foundation. #
|
||||
# We created this package for both routine data analysis and academic #
|
||||
# research and it was publicly released in the hope that it will be #
|
||||
# useful, but it comes WITHOUT ANY WARRANTY OR LIABILITY. #
|
||||
# #
|
||||
# Visit our website for the full manual and a complete tutorial about #
|
||||
# how to conduct AMR data analysis: https://amr-for-r.org #
|
||||
# ==================================================================== #
|
||||
|
||||
# Tests of the `microorganisms` data set and the stability of MO codes.
|
||||
# The rules themselves are in helper-microorganisms.R, which is also used by the taxonomy build.
|
||||
#
|
||||
# KNOWN DEFECTS: the data set on the main branch of October 2026 contains defects that will be solved by the
|
||||
# next taxonomy rebuild (e.g. Trichophyton as a bacterium, issue #309). These are listed in
|
||||
# microorganisms_known_defects.csv, so that these tests fail on every NEW defect, while the current ones are
|
||||
# visible. This list may only become shorter, and must be empty after the rebuild (then delete the file).
|
||||
|
||||
known_defects <- function() {
|
||||
file <- test_path("microorganisms_known_defects.csv")
|
||||
if (!file.exists(file)) {
|
||||
return(data.frame(rule = character(0), record = character(0)))
|
||||
}
|
||||
utils::read.csv(file, colClasses = "character", na.strings = character(0))
|
||||
}
|
||||
new_defects <- function(rule, records) {
|
||||
setdiff(records, known_defects()$record[known_defects()$rule == rule])
|
||||
}
|
||||
|
||||
# Sentinel organisms: clinically important taxa with properties that must not change unnoticed.
|
||||
# NA means: not checked.
|
||||
sentinels <- data.frame(
|
||||
fullname = c(
|
||||
"Escherichia coli", "Klebsiella pneumoniae", "Pseudomonas aeruginosa", "Acinetobacter baumannii",
|
||||
"Haemophilus influenzae", "Neisseria gonorrhoeae", "Staphylococcus aureus", "Streptococcus pneumoniae",
|
||||
"Enterococcus faecium", "Clostridioides difficile", "Mycobacterium tuberculosis", "Listeria monocytogenes",
|
||||
"Tropheryma whipplei", "Candida albicans", "Candidozyma auris", "Nakaseomyces glabratus",
|
||||
"Aspergillus fumigatus", "Cryptococcus neoformans", "Pneumocystis jirovecii", "Trichophyton rubrum",
|
||||
"Mucor", "Necator americanus", "Ascaris lumbricoides", "Echinococcus granulosus",
|
||||
"Capillaria", "Plasmodium falciparum", "Leishmania donovani", "Trypanosoma cruzi",
|
||||
"Giardia", "Toxoplasma gondii"
|
||||
),
|
||||
domain = c(
|
||||
rep("Bacteria", 13), rep("Fungi", 8), rep("Animalia", 4),
|
||||
"Protozoa", "Protozoa", "Protozoa", "Protozoa", NA
|
||||
),
|
||||
gramstain = c(
|
||||
rep("Gram-negative", 6), rep("Gram-positive", 7), rep(NA, 17)
|
||||
),
|
||||
family = c(
|
||||
"Enterobacteriaceae", "Enterobacteriaceae", "Pseudomonadaceae", "Moraxellaceae", "Pasteurellaceae",
|
||||
"Neisseriaceae", "Staphylococcaceae", "Streptococcaceae", "Enterococcaceae", "Peptostreptococcaceae",
|
||||
"Mycobacteriaceae", "Listeriaceae", NA, NA, NA, NA,
|
||||
"Aspergillaceae", NA, NA, "Arthrodermataceae", "Mucoraceae", NA, NA, NA,
|
||||
NA, NA, NA, NA, NA, NA
|
||||
),
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
sentinel_defects <- function() {
|
||||
i <- match(sentinels$fullname, microorganisms$fullname)
|
||||
ok <- !is.na(i)
|
||||
# (paste() of an empty vector would return one element, hence the use of sprintf())
|
||||
out <- sprintf("%s is missing", sentinels$fullname[!ok])
|
||||
check <- function(property, actual) {
|
||||
expected <- sentinels[[property]]
|
||||
wrong <- which(ok & !is.na(expected) & actual != expected)
|
||||
sprintf("%s %s is %s instead of %s", sentinels$fullname[wrong], rep(property, length(wrong)), actual[wrong], expected[wrong])
|
||||
}
|
||||
out <- c(out, check("domain", microorganisms$domain[i]))
|
||||
out <- c(out, check("family", microorganisms$family[i]))
|
||||
gramstain <- rep(NA_character_, length(i))
|
||||
gramstain[ok] <- suppressWarnings(mo_gramstain(microorganisms$mo[i[ok]], language = NULL))
|
||||
out <- c(out, check("gramstain", gramstain))
|
||||
# the code prefix must always match the domain
|
||||
prefix_ok <- startsWith(as.character(microorganisms$mo[i]), paste0(MO_DOMAIN_PREFIX[sentinels$domain], "_"))
|
||||
c(out, sprintf("%s has a code with the wrong prefix", sentinels$fullname[which(ok & !is.na(sentinels$domain) & !prefix_ok)]))
|
||||
}
|
||||
|
||||
test_that("microorganisms: integrity rules", {
|
||||
skip_on_cran()
|
||||
issues <- mo_integrity_issues(microorganisms, registry = read_mo_registry(), renames = read_mo_renames())
|
||||
for (rule in names(issues)) {
|
||||
defects <- new_defects(rule, issues[[rule]])
|
||||
expect_true(length(defects) == 0,
|
||||
info = paste0("Rule '", rule, "' is broken by: ", paste(utils::head(defects, 25), collapse = ", "))
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
test_that("microorganisms: known defects are no longer than needed", {
|
||||
skip_on_cran()
|
||||
# a known defect that is solved must be removed from the list, so that it cannot come back unnoticed
|
||||
kd <- known_defects()
|
||||
skip_if(nrow(kd) == 0)
|
||||
issues <- mo_integrity_issues(microorganisms, registry = read_mo_registry(), renames = read_mo_renames())
|
||||
current <- c(
|
||||
unlist(lapply(names(issues), function(rule) paste(rule, issues[[rule]], sep = "|"))),
|
||||
paste("sentinel", sentinel_defects(), sep = "|")
|
||||
)
|
||||
solved <- setdiff(paste(kd$rule, kd$record, sep = "|"), current)
|
||||
expect_true(length(solved) == 0,
|
||||
info = paste0("Solved, remove from microorganisms_known_defects.csv: ", paste(utils::head(solved, 25), collapse = ", "))
|
||||
)
|
||||
})
|
||||
|
||||
test_that("microorganisms: sentinel organisms", {
|
||||
skip_on_cran()
|
||||
defects <- new_defects("sentinel", sentinel_defects())
|
||||
expect_true(length(defects) == 0, info = paste(defects, collapse = "; "))
|
||||
})
|
||||
|
||||
test_that("MO codes: every code of every release since v2.0.0 still means the same taxon", {
|
||||
skip_on_cran()
|
||||
registry <- read_mo_registry()
|
||||
skip_if(is.null(registry), "MO code registry not available (only in the source repository)")
|
||||
renames <- read_mo_renames()
|
||||
|
||||
# the registry itself
|
||||
expect_false(anyDuplicated(registry$mo) > 0)
|
||||
expect_true(all(numeric_version(sub("^v", "", registry$first_release)) >= "2.0.0"))
|
||||
expect_true(all(numeric_version(sub("^v", "", registry$first_release)) <= numeric_version(sub("^v", "", registry$last_release))))
|
||||
|
||||
# every registered code must lead to its own taxon (or to NA if the taxon is not in the data anymore)
|
||||
expected_name <- registry$fullname
|
||||
renamed <- match(registry$mo, renames$mo)
|
||||
expected_name[!is.na(renamed)] <- renames$new_name[renamed[!is.na(renamed)]]
|
||||
# deliberate in as.mo() since v2.0.0: the kingdom/domain Fungi returns the code for an unknown fungus
|
||||
expected_name[registry$mo == "F_[KNG]_FUNGI"] <- "(unknown fungus)"
|
||||
result <- suppressWarnings(as.mo(registry$mo, keep_synonyms = TRUE, info = FALSE))
|
||||
result_name <- microorganisms$fullname[match(result, microorganisms$mo)]
|
||||
wrong <- !is.na(result) & mo_name_without_suffix(result_name) != mo_name_without_suffix(expected_name)
|
||||
defects <- new_defects("registered_code_with_other_taxon", paste0(registry$mo[wrong], " (", result_name[wrong], ")"))
|
||||
expect_true(length(defects) == 0,
|
||||
info = paste0("Registered codes leading to another taxon: ", paste(utils::head(defects, 25), collapse = ", "))
|
||||
)
|
||||
|
||||
# the internal lookup table must be up to date with the registry
|
||||
retired <- registry$mo[!registry$mo %in% microorganisms$mo]
|
||||
expect_setequal(AMR:::MO_RETIRED_CODES$old_mo, retired)
|
||||
})
|
||||
|
||||
test_that("MO codes: codes of earlier releases are translated, never guessed", {
|
||||
skip_on_cran()
|
||||
retired <- AMR:::MO_RETIRED_CODES
|
||||
expect_false(any(retired$old_mo %in% microorganisms$mo))
|
||||
expect_true(all(retired$mo[!is.na(retired$mo)] %in% microorganisms$mo))
|
||||
# every translated code leads to a record with the registered name
|
||||
translated <- retired[!is.na(retired$mo), , drop = FALSE]
|
||||
expect_identical(
|
||||
as.character(suppressWarnings(as.mo(translated$old_mo, keep_synonyms = TRUE, info = FALSE))),
|
||||
translated$mo
|
||||
)
|
||||
expect_identical(
|
||||
mo_name_without_suffix(microorganisms$fullname[match(translated$mo, microorganisms$mo)]),
|
||||
mo_name_without_suffix(translated$fullname)
|
||||
)
|
||||
|
||||
# regression: in October 2026, the code of Trichophyton (F_TRCHP, v3.0.1) was guessed as Talaromyces
|
||||
expect_warning(x <- as.mo("F_TRCHP_RBRM", info = FALSE), "earlier AMR package version")
|
||||
expect_identical(mo_name(x, language = NULL), "Trichophyton rubrum")
|
||||
expect_identical(suppressWarnings(mo_genus("F_TRCHP", language = NULL)), "Trichophyton")
|
||||
|
||||
# codes from before v2.0.0 and unknown codes are never guessed (decision by Matthijs S. Berends, 3 October 2026)
|
||||
expect_warning(x <- as.mo("B_ESCH_COL", info = FALSE), "not supported")
|
||||
expect_true(is.na(x))
|
||||
expect_warning(x <- as.mo(c("B_ESCHR_COL", "B_ESCHR_COLI"), info = FALSE), "unknown MO code")
|
||||
expect_identical(as.character(x), c(NA, "B_ESCHR_COLI"))
|
||||
|
||||
# regression: until October 2026, a MycoBank target that was not in the data set replaced a valid GBIF target by NA
|
||||
expect_identical(mo_current("Chaetomium abuense"), "Amesia nigricolor")
|
||||
# and a synonym without any known current name is not 'renamed' to itself
|
||||
no_target <- mo_synonyms_without_current_name(microorganisms)
|
||||
expect_true(all(is.na(AMR:::synonym_mo_to_accepted_mo(no_target$mo[is.na(no_target$chain_ends_at)], fill_in_accepted = FALSE))))
|
||||
|
||||
# current codes are returned as they are, without warnings
|
||||
expect_silent(x <- as.mo(c("B_ESCHR_COLI", "B_STPHY_AURS", NA), info = FALSE))
|
||||
expect_identical(as.character(x), c("B_ESCHR_COLI", "B_STPHY_AURS", NA))
|
||||
})
|
||||
@@ -255,10 +255,10 @@ test_that("test-mo.R", {
|
||||
# # expect_warning(as.mo("TestingOwnID", reference_df = NULL))
|
||||
expect_error(as.mo("E. coli", reference_df = data.frame(mycol = "TestingOwnID")))
|
||||
|
||||
# combination of existing mo and other code
|
||||
# combination of an unknown MO code and other code: MO codes are never guessed (see test-data-microorganisms.R)
|
||||
expect_identical(
|
||||
suppressWarnings(as.character(as.mo(c("B_ESCHR_COL", "ESCCOL")))),
|
||||
c("B_ESCHR_COLI", "B_ESCHR_COLI")
|
||||
c(NA, "B_ESCHR_COLI")
|
||||
)
|
||||
|
||||
# from different sources
|
||||
|
||||
Reference in new issue
Block a user