diff --git a/DESCRIPTION b/DESCRIPTION index 7c259842..b7a7e584 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,6 +1,6 @@ Package: AMR -Version: 0.5.0.9021 -Date: 2019-03-09 +Version: 0.5.0.9022 +Date: 2019-03-12 Title: Antimicrobial Resistance Analysis Authors@R: c( person( diff --git a/NEWS.md b/NEWS.md index a8cedcf8..77b96a89 100755 --- a/NEWS.md +++ b/NEWS.md @@ -24,7 +24,7 @@ We've got a new website: [https://msberends.gitlab.io/AMR](https://msberends.git * Support for data from [WHONET](https://whonet.org/) and [EARS-Net](https://ecdc.europa.eu/en/about-us/partnerships-and-networks/disease-and-laboratory-networks/ears-net) (European Antimicrobial Resistance Surveillance Network): * Exported files from WHONET can be read and used in this package. For functions like `first_isolate()` and `eucast_rules()`, all parameters will be filled in automatically. * This package now knows all antibiotic abbrevations by EARS-Net (which are also being used by WHONET) - the `antibiotics` data set now contains a column `ears_net`. - * The function `as.mo()` now knows all WHONET species abbreviations too, because more than 1,600 microbial abbreviations were added to the `microorganisms.codes` data set. + * The function `as.mo()` now knows all WHONET species abbreviations too, because almost 2,000 microbial abbreviations were added to the `microorganisms.codes` data set. * New filters for antimicrobial classes. Use these functions to filter isolates on results in one of more antibiotics from a specific class: ```r filter_aminoglycosides() @@ -100,8 +100,29 @@ We've got a new website: [https://msberends.gitlab.io/AMR](https://msberends.git * Functions `atc_ddd()` and `atc_groups()` have been renamed `atc_online_ddd()` and `atc_online_groups()`. The old functions are deprecated and will be removed in a future version. * Function `guess_mo()` is now deprecated in favour of `as.mo()` and will be removed in future versions * Function `guess_atc()` is now deprecated in favour of `as.atc()` and will be removed in future versions -* Improvements for `as.mo()`:\ - * Incoercible results will now be considered 'unknown', MO code `UNKNOWN`. Properties of these will be translated on foreign systems in all language already previously supported: German, Dutch, French, Italian, Spanish and Portuguese: +* Improvements for `as.mo()`: + * Now handles incorrect spelling like `i` instead of `y` and `f` instead of `ph`: + ```r + # mo_fullname() uses as.mo() internally + + mo_fullname("Sthafilokockus aaureuz") + #> [1] "Staphylococcus aureus" + + mo_fullname("S. klossi") + #> [1] "Staphylococcus kloosii" + ``` + * Uncertainty of the algorithm is now divided into four levels, 0 to 3, where the default `allow_uncertain = TRUE` is equal to uncertainty level 2. Run `?as.mo` for more info about these levels. + ```r + # equal: + as.mo(..., allow_uncertain = TRUE) + as.mo(..., allow_uncertain = 2) + + # also equal: + as.mo(..., allow_uncertain = FALSE) + as.mo(..., allow_uncertain = 0) + ``` + Using `as.mo(..., allow_uncertain = 3)` could lead to very unreliable results. + * Incoercible results will now be considered 'unknown', MO code `UNKNOWN`. On foreign systems, properties of these will be translated to all languages already previously supported: German, Dutch, French, Italian, Spanish and Portuguese: ```r mo_genus("qwerty", language = "es") # Warning: @@ -164,6 +185,7 @@ We've got a new website: [https://msberends.gitlab.io/AMR](https://msberends.git * Automatic parameter filling for `mdro()`, `key_antibiotics()` and `eucast_rules()` * Updated examples for resistance prediction (`resistance_predict()` function) * Fix for `as.mic()` to support more values ending in (several) zeroes +* if using different lengths of pattern and x in `%like%`, it will now return the call #### Other * Updated licence text to emphasise GPL 2.0 and that this is an R package. diff --git a/R/data.R b/R/data.R index 3bfacf79..ace145ef 100755 --- a/R/data.R +++ b/R/data.R @@ -188,7 +188,7 @@ catalogue_of_life <- list( #' Translation table for microorganism codes #' #' A data set containing commonly used codes for microorganisms, from laboratory systems and WHONET. Define your own with \code{\link{set_mo_source}}. -#' @format A \code{\link{data.frame}} with 4,731 observations and 2 variables: +#' @format A \code{\link{data.frame}} with 5,171 observations and 2 variables: #' \describe{ #' \item{\code{certe}}{Commonly used code of a microorganism} #' \item{\code{mo}}{ID of the microorganism in the \code{\link{microorganisms}} data set} diff --git a/R/like.R b/R/like.R index 7e3f4d08..f606c986 100755 --- a/R/like.R +++ b/R/like.R @@ -56,7 +56,7 @@ like <- function(x, pattern) { if (length(pattern) > 1) { if (length(x) != length(pattern)) { pattern <- pattern[1] - warning('only the first element of argument `pattern` used for `%like%`', call. = FALSE) + warning('only the first element of argument `pattern` used for `%like%`', call. = TRUE) } else { # x and pattern are of same length, so items with each other res <- vector(length = length(pattern)) diff --git a/R/mo.R b/R/mo.R index 5cfc5b9b..953b6aac 100755 --- a/R/mo.R +++ b/R/mo.R @@ -21,7 +21,7 @@ #' Transform to microorganism ID #' -#' Use this function to determine a valid microorganism ID (\code{mo}). Determination is done using intelligent rules and the complete taxonomic kingdoms Archaea, Bacteria, Protozoa, Viruses and most microbial species from the kingdom Fungi (see Source), so the input can be almost anything: a full name (like \code{"Staphylococcus aureus"}), an abbreviated name (like \code{"S. aureus"}), an abbreviation known in the field (like \code{"MRSA"}), or just a genus. You could also \code{\link{select}} a genus and species column, zie Examples. +#' Use this function to determine a valid microorganism ID (\code{mo}). Determination is done using intelligent rules and the complete taxonomic kingdoms Bacteria, Chromista, Protozoa, Archaea, Viruses, and most microbial species from the kingdom Fungi (see Source). The input can be almost anything: a full name (like \code{"Staphylococcus aureus"}), an abbreviated name (like \code{"S. aureus"}), an abbreviation known in the field (like \code{"MRSA"}), or just a genus. Please see Examples. #' @param x a character vector or a \code{data.frame} with one or two columns #' @param Becker a logical to indicate whether \emph{Staphylococci} should be categorised into Coagulase Negative \emph{Staphylococci} ("CoNS") and Coagulase Positive \emph{Staphylococci} ("CoPS") instead of their own species, according to Karsten Becker \emph{et al.} [1]. #' @@ -29,7 +29,7 @@ #' @param Lancefield a logical to indicate whether beta-haemolytic \emph{Streptococci} should be categorised into Lancefield groups instead of their own species, according to Rebecca C. Lancefield [2]. These \emph{Streptococci} will be categorised in their first group, e.g. \emph{Streptococcus dysgalactiae} will be group C, although officially it was also categorised into groups G and L. #' #' This excludes \emph{Enterococci} at default (who are in group D), use \code{Lancefield = "all"} to also categorise all \emph{Enterococci} as group D. -#' @param allow_uncertain a logical to indicate whether the input should be checked for less possible results, see Details +#' @param allow_uncertain a logical (\code{TRUE} or \code{FALSE}) or a value between 0 and 3 to indicate whether the input should be checked for less possible results, see Details #' @param reference_df a \code{data.frame} to use for extra reference when translating \code{x} to a valid \code{mo}. See \code{\link{set_mo_source}} and \code{\link{get_mo_source}} to automate the usage of your own codes (e.g. used in your analysis or organisation). #' @rdname as.mo #' @aliases mo @@ -58,9 +58,9 @@ #' \strong{Intelligent rules} \cr #' This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order: #' \itemize{ -#' \item{Taxonomic kingdom: it first searches in Bacteria, then Fungi, then Protozoa} -#' \item{Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see section \emph{Microbial prevalence of pathogens in humans})} #' \item{Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations} +#' \item{Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see \emph{Microbial prevalence of pathogens in humans} below)} +#' \item{Taxonomic kingdom: it first searches in Bacteria/Chromista, then Fungi, then Protozoa, then Viruses} #' \item{Breakdown of input values: from here it starts to breakdown input values to find possible matches} #' } #' @@ -73,15 +73,19 @@ #' This means that looking up human pathogenic microorganisms takes less time than looking up human non-pathogenic microorganisms. #' #' \strong{Uncertain results} \cr -#' When using \code{allow_uncertain = TRUE} (which is the default setting), it will use additional rules if all previous rules failed to get valid results. These are: +#' The algorithm can additionally use three different levels of uncertainty to guess valid results. The default is \code{allow_uncertain = TRUE}, which is uqual to uncertainty level 2. Using \code{allow_uncertain = FALSE} will skip all of these additional rules: #' \itemize{ -#' \item{It tries to look for previously accepted (but now invalid) taxonomic names} -#' \item{It strips off values between brackets and the brackets itself, and re-evaluates the input with all previous rules} -#' \item{It strips off words from the end one by one and re-evaluates the input with all previous rules} -#' \item{It strips off words from the start one by one and re-evaluates the input with all previous rules} -#' \item{It tries to look for some manual changes which are not (yet) published to the Catalogue of Life (like \emph{Propionibacterium} being \emph{Cutibacterium})} +#' \item{(uncertainty level 1): It tries to look for only matching genera} +#' \item{(uncertainty level 1): It tries to look for previously accepted (but now invalid) taxonomic names} +#' \item{(uncertainty level 1): It tries to look for some manual changes which are not (yet) published to the Catalogue of Life (like \emph{Propionibacterium} being \emph{Cutibacterium})} +#' \item{(uncertainty level 2): It strips off values between brackets and the brackets itself, and re-evaluates the input with all previous rules} +#' \item{(uncertainty level 2): It strips off words from the end one by one and re-evaluates the input with all previous rules} +#' \item{(uncertainty level 3): It strips off words from the start one by one and re-evaluates the input with all previous rules} +#' \item{(uncertainty level 3): It tries any part of the name} #' } #' +#' You can also use e.g. \code{as.mo(..., allow_uncertain = 1)} to only allow up to level 1 uncertainty. +#' #' Examples: #' \itemize{ #' \item{\code{"Streptococcus group B (known as S. agalactiae)"}. The text between brackets will be removed and a warning will be thrown that the result \emph{Streptococcus group B} (\code{B_STRPT_GRB}) needs review.} @@ -96,7 +100,7 @@ #' Use \code{mo_renamed()} to get a vector with all values that could be coerced based on an old, previously accepted taxonomic name. #' #' \strong{Microbial prevalence of pathogens in humans} \cr -#' The intelligent rules takes into account microbial prevalence of pathogens in humans. It uses three groups and every (sub)species is in the group it matches first. These groups are: +#' The intelligent rules takes into account microbial prevalence of pathogens in humans. It uses three groups and all (sub)species are in only one group. These groups are: #' \itemize{ #' \item{1 (most prevalent): class is Gammaproteobacteria \strong{or} genus is one of: \emph{Enterococcus}, \emph{Staphylococcus}, \emph{Streptococcus}.} #' \item{2: phylum is one of: Proteobacteria, Firmicutes, Actinobacteria, Sarcomastigophora \strong{or} genus is one of: \emph{Aspergillus}, \emph{Bacteroides}, \emph{Candida}, \emph{Capnocytophaga}, \emph{Chryseobacterium}, \emph{Cryptococcus}, \emph{Elisabethkingia}, \emph{Flavobacterium}, \emph{Fusobacterium}, \emph{Giardia}, \emph{Leptotrichia}, \emph{Mycoplasma}, \emph{Prevotella}, \emph{Rhodotorula}, \emph{Treponema}, \emph{Trichophyton}, \emph{Ureaplasma}.} @@ -130,6 +134,7 @@ #' as.mo("S aureus") #' as.mo("Staphylococcus aureus") #' as.mo("Staphylococcus aureus (MRSA)") +#' as.mo("Sthafilokkockus aaureuz") # handles incorrect spelling #' as.mo("MRSA") # Methicillin Resistant S. aureus #' as.mo("VISA") # Vancomycin Intermediate S. aureus #' as.mo("VRSA") # Vancomycin Resistant S. aureus @@ -202,8 +207,8 @@ as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, allow_uncertain = TRUE, ) } else if (all(x %in% AMR::microorganisms$mo) - & isFALSE(Becker) - & isFALSE(Lancefield)) { + & isFALSE(Becker) + & isFALSE(Lancefield)) { y <- x } else if (all(tolower(x) %in% microorganismsDT$fullname_lower) @@ -284,6 +289,15 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, fullname = character(0), mo = character(0)) failures <- character(0) + if (isTRUE(allow_uncertain)) { + # default to uncertainty level 2 + allow_uncertain <- 2 + } else { + allow_uncertain <- as.integer(allow_uncertain) + if (!allow_uncertain %in% c(0:3)) { + stop("`allow_uncertain` must be a number between 0 (none) and 3 (all), or TRUE (= 2) or FALSE (= 0).", call. = FALSE) + } + } x_input <- x # already strip leading and trailing spaces x <- trimws(x, which = "both") @@ -387,6 +401,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, # remove spp and species x <- trimws(gsub(" +(spp.?|ssp.?|sp.? |ss ?.?|subsp.?|subspecies|biovar |serovar |species)", " ", x_backup, ignore.case = TRUE), which = "both") + x_backup_without_spp <- x x_species <- paste(x, "species") # translate to English for supported languages of mo_property x <- gsub("(Gruppe|gruppe|groep|grupo|gruppo|groupe)", "group", x, ignore.case = TRUE) @@ -400,12 +415,21 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, x <- gsub("(alpha|beta|gamma) ha?emoly", "\\1-haemoly", x) # remove genus as first word x <- gsub("^Genus ", "", x) + # allow characters that resemble others + x <- gsub("[iy]+", "[iy]+", x, ignore.case = TRUE) + x <- gsub("[sz]+", "[sz]+", x, ignore.case = TRUE) + x <- gsub("(c|k|q|qu)+", "(c|k|q|qu)+", x, ignore.case = TRUE) + x <- gsub("(ph|f|v)+", "(ph|f|v)+", x, ignore.case = TRUE) + x <- gsub("(th|t)+", "(th|t)+", x, ignore.case = TRUE) + x <- gsub("a+", "a+", x, ignore.case = TRUE) + x <- gsub("e+", "e+", x, ignore.case = TRUE) + x <- gsub("o+", "o+", x, ignore.case = TRUE) # but spaces before and after should be omitted x <- trimws(x, which = "both") x_trimmed <- x x_trimmed_species <- paste(x_trimmed, "species") - x_trimmed_without_group <- gsub(" group$", "", x_trimmed, ignore.case = TRUE) + x_trimmed_without_group <- gsub(" gro.u.p$", "", x_trimmed, ignore.case = TRUE) # remove last part from "-" or "/" x_trimmed_without_group <- gsub("(.*)[-/].*", "\\1", x_trimmed_without_group) # replace space and dot by regex sign @@ -423,6 +447,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, # cat(paste0('x_withspaces_end_only "', x_withspaces_end_only, '"\n')) # cat(paste0('x_withspaces_start_end "', x_withspaces_start_end, '"\n')) # cat(paste0('x_backup "', x_backup, '"\n')) + # cat(paste0('x_backup_without_spp "', x_backup_without_spp, '"\n')) # cat(paste0('x_trimmed "', x_trimmed, '"\n')) # cat(paste0('x_trimmed_species "', x_trimmed_species, '"\n')) # cat(paste0('x_trimmed_without_group "', x_trimmed_without_group, '"\n')) @@ -440,12 +465,19 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, next } - if (any(x_trimmed[i] %in% c(NA, "", "xxx", "con"))) { + found <- microorganismsDT[fullname_lower %in% tolower(c(x_backup[i], x_backup_without_spp[i])), ..property][[1]] + # most probable: is exact match in fullname + if (length(found) > 0) { + x[i] <- found[1L] + next + } + + if (any(x_backup_without_spp[i] %in% c(NA, "", "xxx", "con"))) { x[i] <- NA_character_ next } - if (tolower(x_trimmed[i]) %in% c("other", "none", "unknown")) { + if (tolower(x_backup_without_spp[i]) %in% c("other", "none", "unknown")) { # empty and nonsense values, ignore without warning x[i] <- microorganismsDT[mo == "UNKNOWN", ..property][[1]] next @@ -472,7 +504,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, next } - if (x_trimmed[i] %like% "virus") { + if (x_backup_without_spp[i] %like% "virus") { # there is no fullname like virus, so don't try to coerce it x[i] <- microorganismsDT[mo == "UNKNOWN", ..property][[1]] failures <- c(failures, x_backup[i]) @@ -481,100 +513,100 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, # translate known trivial abbreviations to genus + species ---- if (!is.na(x_trimmed[i])) { - if (toupper(x_trimmed[i]) %in% c('MRSA', 'MSSA', 'VISA', 'VRSA')) { + if (toupper(x_backup_without_spp[i]) %in% c('MRSA', 'MSSA', 'VISA', 'VRSA')) { x[i] <- microorganismsDT[mo == 'B_STPHY_AUR', ..property][[1]][1L] next } - if (toupper(x_trimmed[i]) %in% c('MRSE', 'MSSE')) { + if (toupper(x_backup_without_spp[i]) %in% c('MRSE', 'MSSE')) { x[i] <- microorganismsDT[mo == 'B_STPHY_EPI', ..property][[1]][1L] next } - if (toupper(x_trimmed[i]) == "VRE" - | x_trimmed[i] %like% '(enterococci|enterokok|enterococo)[a-z]*?$') { + if (toupper(x_backup_without_spp[i]) == "VRE" + | x_backup_without_spp[i] %like% '(enterococci|enterokok|enterococo)[a-z]*?$') { x[i] <- microorganismsDT[mo == 'B_ENTRC', ..property][[1]][1L] next } - if (toupper(x_trimmed[i]) %in% c("EHEC", "EPEC", "EIEC", "STEC", "ATEC")) { + if (toupper(x_backup_without_spp[i]) %in% c("EHEC", "EPEC", "EIEC", "STEC", "ATEC")) { x[i] <- microorganismsDT[mo == 'B_ESCHR_COL', ..property][[1]][1L] next } - if (toupper(x_trimmed[i]) == 'MRPA') { + if (toupper(x_backup_without_spp[i]) == 'MRPA') { # multi resistant P. aeruginosa x[i] <- microorganismsDT[mo == 'B_PSDMN_AER', ..property][[1]][1L] next } - if (toupper(x_trimmed[i]) == 'CRS' - | toupper(x_trimmed[i]) == 'CRSM') { + if (toupper(x_backup_without_spp[i]) == 'CRS' + | toupper(x_backup_without_spp[i]) == 'CRSM') { # co-trim resistant S. maltophilia x[i] <- microorganismsDT[mo == 'B_STNTR_MAL', ..property][[1]][1L] next } - if (toupper(x_trimmed[i]) %in% c('PISP', 'PRSP', 'VISP', 'VRSP')) { + if (toupper(x_backup_without_spp[i]) %in% c('PISP', 'PRSP', 'VISP', 'VRSP')) { # peni I, peni R, vanco I, vanco R: S. pneumoniae x[i] <- microorganismsDT[mo == 'B_STRPT_PNE', ..property][[1]][1L] next } - if (x_trimmed[i] %like% '^G[ABCDFGHK]S$') { + if (x_backup_without_spp[i] %like% '^G[ABCDFGHK]S$') { # Streptococci, like GBS = Group B Streptococci (B_STRPT_GRB) - x[i] <- microorganismsDT[mo == gsub("G([ABCDFGHK])S", "B_STRPT_GR\\1", x_trimmed[i], ignore.case = TRUE), ..property][[1]][1L] + x[i] <- microorganismsDT[mo == gsub("G([ABCDFGHK])S", "B_STRPT_GR\\1", x_backup_without_spp[i], ignore.case = TRUE), ..property][[1]][1L] next } - if (x_trimmed[i] %like% '(streptococ|streptokok).* [ABCDFGHK]$') { + if (x_backup_without_spp[i] %like% '(streptococ|streptokok).* [ABCDFGHK]$') { # Streptococci in different languages, like "estreptococos grupo B" - x[i] <- microorganismsDT[mo == gsub(".*(streptococ|streptokok|estreptococ).* ([ABCDFGHK])$", "B_STRPT_GR\\2", x_trimmed[i], ignore.case = TRUE), ..property][[1]][1L] + x[i] <- microorganismsDT[mo == gsub(".*(streptococ|streptokok|estreptococ).* ([ABCDFGHK])$", "B_STRPT_GR\\2", x_backup_without_spp[i], ignore.case = TRUE), ..property][[1]][1L] next } - if (x_trimmed[i] %like% 'group [ABCDFGHK] (streptococ|streptokok|estreptococ)') { + if (x_backup_without_spp[i] %like% 'group [ABCDFGHK] (streptococ|streptokok|estreptococ)') { # Streptococci in different languages, like "Group A Streptococci" - x[i] <- microorganismsDT[mo == gsub(".*group ([ABCDFGHK]) (streptococ|streptokok|estreptococ).*", "B_STRPT_GR\\1", x_trimmed[i], ignore.case = TRUE), ..property][[1]][1L] + x[i] <- microorganismsDT[mo == gsub(".*group ([ABCDFGHK]) (streptococ|streptokok|estreptococ).*", "B_STRPT_GR\\1", x_backup_without_spp[i], ignore.case = TRUE), ..property][[1]][1L] next } # CoNS/CoPS in different languages (support for German, Dutch, Spanish, Portuguese) ---- - if (x[i] %like% '[ck]oagulas[ea] negatie?[vf]' + if (x_backup_without_spp[i] %like% '[ck]oagulas[ea] negatie?[vf]' | x_trimmed[i] %like% '[ck]oagulas[ea] negatie?[vf]' - | x[i] %like% '[ck]o?ns[^a-z]?$') { + | x_backup_without_spp[i] %like% '[ck]o?ns[^a-z]?$') { # coerce S. coagulase negative x[i] <- microorganismsDT[mo == 'B_STPHY_CNS', ..property][[1]][1L] next } - if (x[i] %like% '[ck]oagulas[ea] positie?[vf]' + if (x_backup_without_spp[i] %like% '[ck]oagulas[ea] positie?[vf]' | x_trimmed[i] %like% '[ck]oagulas[ea] positie?[vf]' - | x[i] %like% '[ck]o?ps[^a-z]?$') { + | x_backup_without_spp[i] %like% '[ck]o?ps[^a-z]?$') { # coerce S. coagulase positive x[i] <- microorganismsDT[mo == 'B_STPHY_CPS', ..property][[1]][1L] next } - if (x[i] %like% 'gram[ -]?neg.*' + if (x_backup_without_spp[i] %like% 'gram[ -]?neg.*' | x_trimmed[i] %like% 'gram[ -]?neg.*') { - # coerce S. coagulase positive + # coerce Gram negatives x[i] <- microorganismsDT[mo == 'B_GRAMN', ..property][[1]][1L] next } - if (x[i] %like% 'gram[ -]?pos.*' + if (x_backup_without_spp[i] %like% 'gram[ -]?pos.*' | x_trimmed[i] %like% 'gram[ -]?pos.*') { - # coerce S. coagulase positive + # coerce Gram positives x[i] <- microorganismsDT[mo == 'B_GRAMP', ..property][[1]][1L] next } - if (grepl("[sS]almonella [A-Z][a-z]+ ?.*", x_trimmed[i], ignore.case = FALSE)) { - if (x_trimmed[i] %like% "Salmonella group") { + if (grepl("[sS]almonella [A-Z][a-z]+ ?.*", x_backup_without_spp[i], ignore.case = FALSE)) { + if (x_backup_without_spp[i] %like% "Salmonella group") { # Salmonella Group A to Z, just return S. species for now x[i] <- microorganismsDT[mo == 'B_SLMNL', ..property][[1]][1L] - notes <- c(notes, - magenta(paste0("Note: ", - italic("Salmonella"), " ", trimws(gsub("Salmonella", "", x_trimmed[i])), - " was considered ", - italic("Salmonella species"), - " (B_SLMNL)"))) + options(mo_renamed = c(getOption("mo_renamed"), + magenta(paste0("Note: ", + italic("Salmonella"), " ", trimws(gsub("Salmonella", "", x_backup_without_spp[i])), + " was considered ", + italic("Salmonella species"), + " (B_SLMNL)")))) } else { # Salmonella with capital letter species like "Salmonella Goettingen" - they're all S. enterica x[i] <- microorganismsDT[mo == 'B_SLMNL_ENT', ..property][[1]][1L] - notes <- c(notes, - magenta(paste0("Note: ", - italic("Salmonella"), " ", trimws(gsub("Salmonella", "", x_trimmed[i])), - " was considered a subspecies of ", - italic("Salmonella enterica"), - " (B_SLMNL_ENT)"))) + options(mo_renamed = c(getOption("mo_renamed"), + magenta(paste0("Note: ", + italic("Salmonella"), " ", trimws(gsub("Salmonella", "", x_backup_without_spp[i])), + " was considered a subspecies of ", + italic("Salmonella enterica"), + " (B_SLMNL_ENT)")))) } next } @@ -588,8 +620,8 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, x[i] <- found[1L] next } - if (nchar(x_trimmed[i]) >= 6) { - found <- microorganismsDT[fullname_lower %like% paste0(x_withspaces_start_only[i], "[a-z]+ species"), ..property][[1]] + if (nchar(x_backup_without_spp[i]) >= 6) { + found <- microorganismsDT[fullname_lower %like% paste0("^", x_backup_without_spp[i], "[a-z]+"), ..property][[1]] if (length(found) > 0) { x[i] <- found[1L] next @@ -621,7 +653,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, } # allow no codes less than 4 characters long, was already checked for WHONET above - if (nchar(x_trimmed[i]) < 4) { + if (nchar(x_backup_without_spp[i]) < 4) { x[i] <- microorganismsDT[mo == "UNKNOWN", ..property][[1]] failures <- c(failures, x_backup[i]) next @@ -633,22 +665,23 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, c.x_trimmed_without_group, d.x_withspaces_start_end, e.x_withspaces_start_only, - f.x_withspaces_end_only) { + f.x_withspaces_end_only, + g.x_backup_without_spp) { - found <- data_to_check[fullname_lower %in% tolower(c(a.x_backup, b.x_trimmed)), ..property][[1]] - # most probable: is exact match in fullname + # try probable: trimmed version of fullname ---- + found <- data_to_check[fullname_lower %in% tolower(g.x_backup_without_spp), ..property][[1]] if (length(found) > 0) { return(found[1L]) } - - found <- data_to_check[fullname_lower == tolower(c.x_trimmed_without_group), ..property][[1]] - if (length(found) > 0) { + found <- data_to_check[fullname_lower %like% b.x_trimmed + | fullname_lower %like% c.x_trimmed_without_group, ..property][[1]] + if (length(found) > 0 & nchar(g.x_backup_without_spp) >= 6) { return(found[1L]) } # try any match keeping spaces ---- found <- data_to_check[fullname %like% d.x_withspaces_start_end, ..property][[1]] - if (length(found) > 0 & nchar(b.x_trimmed) >= 6) { + if (length(found) > 0 & nchar(g.x_backup_without_spp) >= 6) { return(found[1L]) } @@ -658,7 +691,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, return(found[1L]) } found <- data_to_check[fullname %like% e.x_withspaces_start_only, ..property][[1]] - if (length(found) > 0 & nchar(b.x_trimmed) >= 6) { + if (length(found) > 0 & nchar(g.x_backup_without_spp) >= 6) { return(found[1L]) } @@ -671,12 +704,12 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, # try splitting of characters in the middle and then find ID ---- # only when text length is 6 or lower # like esco = E. coli, klpn = K. pneumoniae, stau = S. aureus, staaur = S. aureus - if (nchar(b.x_trimmed) <= 6) { - x_length <- nchar(b.x_trimmed) + if (nchar(g.x_backup_without_spp) <= 6) { + x_length <- nchar(g.x_backup_without_spp) x_split <- paste0("^", - b.x_trimmed %>% substr(1, x_length / 2), + g.x_backup_without_spp %>% substr(1, x_length / 2), '.* ', - b.x_trimmed %>% substr((x_length / 2) + 1, x_length)) + g.x_backup_without_spp %>% substr((x_length / 2) + 1, x_length)) found <- data_to_check[fullname %like% x_split, ..property][[1]] if (length(found) > 0) { return(found[1L]) @@ -701,7 +734,8 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, c.x_trimmed_without_group = x_trimmed_without_group[i], d.x_withspaces_start_end = x_withspaces_start_end[i], e.x_withspaces_start_only = x_withspaces_start_only[i], - f.x_withspaces_end_only = x_withspaces_end_only[i]) + f.x_withspaces_end_only = x_withspaces_end_only[i], + g.x_backup_without_spp = x_backup_without_spp[i]) if (!empty_result(x[i])) { next } @@ -712,7 +746,8 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, c.x_trimmed_without_group = x_trimmed_without_group[i], d.x_withspaces_start_end = x_withspaces_start_end[i], e.x_withspaces_start_only = x_withspaces_start_only[i], - f.x_withspaces_end_only = x_withspaces_end_only[i]) + f.x_withspaces_end_only = x_withspaces_end_only[i], + g.x_backup_without_spp = x_backup_without_spp[i]) if (!empty_result(x[i])) { next } @@ -723,7 +758,8 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, c.x_trimmed_without_group = x_trimmed_without_group[i], d.x_withspaces_start_end = x_withspaces_start_end[i], e.x_withspaces_start_only = x_withspaces_start_only[i], - f.x_withspaces_end_only = x_withspaces_end_only[i]) + f.x_withspaces_end_only = x_withspaces_end_only[i], + g.x_backup_without_spp = x_backup_without_spp[i]) if (!empty_result(x[i])) { next } @@ -752,31 +788,23 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, } # check for uncertain results ---- - if (allow_uncertain == TRUE) { + uncertain_fn <- function(a.x_backup, +b.x_trimmed, + c.x_withspaces_start_end, +d.x_withspaces_start_only, + f.x_withspaces_end_only, +g.x_backup_without_spp) { - uncertain_fn <- function(a.x_backup, b.x_trimmed, c.x_withspaces_start_end, d.x_withspaces_start_only, f.x_withspaces_end_only) { + if (allow_uncertain == 0) { + # do not allow uncertainties + return(NA_character_) + } - # (1) look for genus only, part of name ---- - if (nchar(b.x_trimmed) > 4 & !b.x_trimmed %like% " ") { - if (!grepl("^[A-Z][a-z]+", b.x_trimmed, ignore.case = FALSE)) { - # not when input is like Genustext, because then Neospora would lead to Actinokineospora - found <- microorganismsDT[fullname_lower %like% paste(b.x_trimmed, "species"), ..property][[1]] - if (length(found) > 0) { - x[i] <- found[1L] - uncertainties <<- rbind(uncertainties, - data.frame(uncertainty = 2, - input = a.x_backup, - fullname = microorganismsDT[mo == found[1L], fullname][[1]], - mo = found[1L])) - return(x) - } - } - } - - # (2) look again for old taxonomic names, now for G. species ---- + if (allow_uncertain >= 1) { + # (1) look again for old taxonomic names, now for G. species ---- found <- microorganisms.oldDT[fullname %like% c.x_withspaces_start_end | fullname %like% d.x_withspaces_start_only] - if (NROW(found) > 0 & nchar(b.x_trimmed) >= 6) { + if (NROW(found) > 0 & nchar(g.x_backup_without_spp) >= 6) { if (property == "ref") { # when property is "ref" (which is the case in mo_ref, mo_authors and mo_year), return the old value, so: # mo_ref("Chlamydia psittaci) = "Page, 1968" (with warning) @@ -798,7 +826,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, return(x) } - # (3) not yet implemented taxonomic changes in Catalogue of Life ---- + # (2) not yet implemented taxonomic changes in Catalogue of Life ---- found <- suppressMessages(suppressWarnings(exec_as.mo(TEMPORARY_TAXONOMY(b.x_trimmed), clear_options = FALSE, allow_uncertain = FALSE))) if (!empty_result(found)) { found_result <- found @@ -810,12 +838,31 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, mo = found_result[1L])) return(found[1L]) } + } + + if (allow_uncertain >= 2) { + # (3) look for genus only, part of name ---- + if (nchar(g.x_backup_without_spp) > 4 & !b.x_trimmed %like% " ") { + if (!grepl("^[A-Z][a-z]+", b.x_trimmed, ignore.case = FALSE)) { + # not when input is like Genustext, because then Neospora would lead to Actinokineospora + found <- microorganismsDT[fullname_lower %like% paste(b.x_trimmed, "species"), ..property][[1]] + if (length(found) > 0) { + x[i] <- found[1L] + uncertainties <<- rbind(uncertainties, + data.frame(uncertainty = 2, + input = a.x_backup, + fullname = microorganismsDT[mo == found[1L], fullname][[1]], + mo = found[1L])) + return(x) + } + } + } # (4) strip values between brackets ---- a.x_backup_stripped <- gsub("( *[(].*[)] *)", " ", a.x_backup) a.x_backup_stripped <- trimws(gsub(" +", " ", a.x_backup_stripped)) found <- suppressMessages(suppressWarnings(exec_as.mo(a.x_backup_stripped, clear_options = FALSE, allow_uncertain = FALSE))) - if (!empty_result(found) & nchar(b.x_trimmed) >= 6) { + if (!empty_result(found) & nchar(g.x_backup_without_spp) >= 6) { found_result <- found found <- microorganismsDT[mo == found, ..property][[1]] uncertainties <<- rbind(uncertainties, @@ -828,26 +875,30 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, # (5) try to strip off one element from end and check the remains ---- x_strip <- a.x_backup %>% strsplit(" ") %>% unlist() - if (length(x_strip) > 1 & nchar(b.x_trimmed) >= 6) { + if (length(x_strip) > 1) { for (i in 1:(length(x_strip) - 1)) { x_strip_collapsed <- paste(x_strip[1:(length(x_strip) - i)], collapse = " ") - found <- suppressMessages(suppressWarnings(exec_as.mo(x_strip_collapsed, clear_options = FALSE, allow_uncertain = FALSE))) - if (!empty_result(found)) { - found_result <- found - found <- microorganismsDT[mo == found, ..property][[1]] - uncertainties <<- rbind(uncertainties, - data.frame(uncertainty = 2, - input = a.x_backup, - fullname = microorganismsDT[mo == found_result[1L], fullname][[1]], - mo = found_result[1L])) - return(found[1L]) + if (nchar(x_strip_collapsed) >= 4) { + found <- suppressMessages(suppressWarnings(exec_as.mo(x_strip_collapsed, clear_options = FALSE, allow_uncertain = FALSE))) + if (!empty_result(found)) { + found_result <- found + found <- microorganismsDT[mo == found, ..property][[1]] + uncertainties <<- rbind(uncertainties, + data.frame(uncertainty = 2, + input = a.x_backup, + fullname = microorganismsDT[mo == found_result[1L], fullname][[1]], + mo = found_result[1L])) + return(found[1L]) + } } } } + } + if (allow_uncertain >= 3) { # (6) try to strip off one element from start and check the remains ---- x_strip <- a.x_backup %>% strsplit(" ") %>% unlist() - if (length(x_strip) > 1 & nchar(b.x_trimmed) >= 6) { + if (length(x_strip) > 1 & nchar(g.x_backup_without_spp) >= 6) { for (i in 2:(length(x_strip))) { x_strip_collapsed <- paste(x_strip[i:length(x_strip)], collapse = " ") found <- suppressMessages(suppressWarnings(exec_as.mo(x_strip_collapsed, clear_options = FALSE, allow_uncertain = FALSE))) @@ -868,7 +919,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, found <- microorganismsDT[fullname %like% f.x_withspaces_end_only] if (nrow(found) > 0) { found_result <- found[["mo"]] - if (!empty_result(found_result)) { + if (!empty_result(found_result) & nchar(g.x_backup_without_spp) >= 6) { found <- microorganismsDT[mo == found_result[1L], ..property][[1]] uncertainties <<- rbind(uncertainties, data.frame(uncertainty = 3, @@ -878,16 +929,21 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, return(found[1L]) } } - - # didn't found in uncertain results too - return(NA_character_) } - x[i] <- uncertain_fn(x_backup[i], x_trimmed[i], x_withspaces_start_end[i], x_withspaces_start_only[i], x_withspaces_end_only[i]) - if (!empty_result(x[i])) { - next - } + # didn't found in uncertain results too + return(NA_character_) } + x[i] <- uncertain_fn(x_backup[i], + x_trimmed[i], + x_withspaces_start_end[i], + x_withspaces_start_only[i], + x_withspaces_end_only[i], + x_backup_without_spp[i]) + if (!empty_result(x[i])) { + next + } + # not found ---- x[i] <- microorganismsDT[mo == "UNKNOWN", ..property][[1]] @@ -899,19 +955,19 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, failures <- failures[!failures %in% c(NA, NULL, NaN)] if (length(failures) > 0 & clear_options == TRUE) { options(mo_failures = sort(unique(failures))) - plural <- c("value", "it", "is") + plural <- c("value", "it", "was") if (n_distinct(failures) > 1) { - plural <- c("values", "them", "are") + plural <- c("values", "them", "were") } total_failures <- length(x_input[x_input %in% failures & !x_input %in% c(NA, NULL, NaN)]) total_n <- length(x_input[!x_input %in% c(NA, NULL, NaN)]) - msg <- paste0("\n", nr2char(n_distinct(failures)), " unique ", plural[1], + msg <- paste0(nr2char(n_distinct(failures)), " unique ", plural[1], " (^= ", percent(total_failures / total_n, round = 1, force_zero = TRUE), ") could not be coerced and ", plural[3], " considered 'unknown'") if (n_distinct(failures) <= 10) { msg <- paste0(msg, ": ", paste('"', unique(failures), '"', sep = "", collapse = ', ')) } - msg <- paste0(msg, ". Use mo_failures() to review ", plural[2], ".") + msg <- paste0(msg, ". Use mo_failures() to review ", plural[2], ". Edit the `allow_uncertain` parameter if needed (see ?as.mo).") warning(red(msg), call. = FALSE, immediate. = TRUE) # thus will always be shown, even if >= warnings @@ -1026,7 +1082,7 @@ exec_as.mo <- function(x, Becker = FALSE, Lancefield = FALSE, } empty_result <- function(x) { - x %in% c(NA, "UNKNOWN") + all(x %in% c(NA, "UNKNOWN")) } TEMPORARY_TAXONOMY <- function(x) { @@ -1124,6 +1180,9 @@ mo_uncertainties <- function() { #' @export #' @noRd print.mo_uncertainties <- function(x, ...) { + if (NROW(x) == 0) { + return(NULL) + } cat(paste0(bold(nrow(x), "unique result(s) guessed with uncertainty:"), "\n(1 = ", green("renamed"), ", 2 = ", yellow("uncertain"), diff --git a/R/zzz.R b/R/zzz.R index 78dccdcf..6d4a87df 100755 --- a/R/zzz.R +++ b/R/zzz.R @@ -25,28 +25,25 @@ backports::import(pkgname) # register data - if (!all(c("microorganismsDT", "microorganisms.oldDT") %in% ls(envir = asNamespace("AMR")))) { + microorganisms.oldDT <- as.data.table(AMR::microorganisms.old) + microorganisms.oldDT$fullname_lower <- tolower(microorganisms.oldDT$fullname) + setkey(microorganisms.oldDT, col_id, fullname) - microorganisms.oldDT <- as.data.table(AMR::microorganisms.old) - microorganisms.oldDT$fullname_lower <- tolower(microorganisms.oldDT$fullname) - setkey(microorganisms.oldDT, col_id, fullname) + assign(x = "microorganisms", + value = make(), + envir = asNamespace("AMR")) - assign(x = "microorganisms", - value = make(), - envir = asNamespace("AMR")) + assign(x = "microorganismsDT", + value = make_DT(), + envir = asNamespace("AMR")) - assign(x = "microorganismsDT", - value = make_DT(), - envir = asNamespace("AMR")) + assign(x = "microorganisms.oldDT", + value = microorganisms.oldDT, + envir = asNamespace("AMR")) - assign(x = "microorganisms.oldDT", - value = microorganisms.oldDT, - envir = asNamespace("AMR")) - - assign(x = "mo_codes_v0.5.0", - value = make_trans_tbl(), - envir = asNamespace("AMR")) - } + assign(x = "mo_codes_v0.5.0", + value = make_trans_tbl(), + envir = asNamespace("AMR")) } #' @importFrom dplyr mutate case_when @@ -88,8 +85,8 @@ make_DT <- function() { microorganismsDT <- as.data.table(make()) microorganismsDT$fullname_lower <- tolower(microorganismsDT$fullname) setkey(microorganismsDT, - kingdom, prevalence, + kingdom, fullname) microorganismsDT } diff --git a/data/microorganisms.codes.rda b/data/microorganisms.codes.rda index e5254a13..8b7b005e 100644 Binary files a/data/microorganisms.codes.rda and b/data/microorganisms.codes.rda differ diff --git a/docs/LICENSE-text.html b/docs/LICENSE-text.html index f4aca6a7..5e593eff 100644 --- a/docs/LICENSE-text.html +++ b/docs/LICENSE-text.html @@ -78,7 +78,7 @@
diff --git a/docs/articles/benchmarks.html b/docs/articles/benchmarks.html index 6848bcaa..9f9c9b39 100644 --- a/docs/articles/benchmarks.html +++ b/docs/articles/benchmarks.html @@ -40,7 +40,7 @@ @@ -192,7 +192,7 @@benchmarks.Rmd
In the table above, all measurements are shown in milliseconds (thousands of seconds). A value of 5 milliseconds means it can determine 200 input values per second. It case of 100 milliseconds, this is only 10 input values per second. The second input is the only one that has to be looked up thoroughly. All the others are known codes (the first one is a WHONET code) or common laboratory codes, or common full organism names like the last one. Full organism names are always preferred.
To achieve this speed, the as.mo
function also takes into account the prevalence of human pathogenic microorganisms. The downside is of course that less prevalent microorganisms will be determined less fast. See this example for the ID of Thermus islandicus (B_THERMS_ISL
), a bug probably never found before in humans:
T.islandicus <- microbenchmark(as.mo("theisl"),
@@ -235,13 +235,13 @@
times = 10)
print(T.islandicus, unit = "ms", signif = 3)
#> Unit: milliseconds
-#> expr min lq mean median uq max neval
-#> as.mo("theisl") 262.0 263.0 284.0 284.0 304 308 10
-#> as.mo("THEISL") 263.0 264.0 293.0 304.0 306 308 10
-#> as.mo("T. islandicus") 142.0 142.0 151.0 143.0 147 187 10
-#> as.mo("T. islandicus") 142.0 142.0 169.0 184.0 185 194 10
-#> as.mo("Thermus islandicus") 67.9 68.1 93.3 90.3 116 130 10
That takes 7.8 times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like Thermus islandicus) are almost fast - these are the most probable input from most data sets.
+#> expr min lq mean median uq max neval +#> as.mo("theisl") 417.0 419.0 450.0 460.0 464.0 474 10 +#> as.mo("THEISL") 415.0 416.0 443.0 458.0 460.0 468 10 +#> as.mo("T. islandicus") 281.0 281.0 299.0 285.0 325.0 352 10 +#> as.mo("T. islandicus") 292.0 298.0 341.0 336.0 340.0 495 10 +#> as.mo("Thermus islandicus") 66.2 66.5 75.5 66.9 68.2 112 10 +That takes 10.9 times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like Thermus islandicus) are almost fast - these are the most probable input from most data sets.
In the figure below, we compare Escherichia coli (which is very common) with Prevotella brevis (which is moderately common) and with Thermus islandicus (which is very uncommon):
par(mar = c(5, 16, 4, 2)) # set more space for left margin text (16)
@@ -286,9 +286,9 @@
times = 10)
print(run_it, unit = "ms", signif = 3)
#> Unit: milliseconds
-#> expr min lq mean median uq max neval
-#> mo_fullname(x) 734 810 840 817 860 973 10
So transforming 500,000 values (!!) of 50 unique values only takes 0.82 seconds (817 ms). You only lose time on your unique input values.
+#> expr min lq mean median uq max neval +#> mo_fullname(x) 794 834 863 844 876 1050 10 +So transforming 500,000 values (!!) of 50 unique values only takes 0.84 seconds (844 ms). You only lose time on your unique input values.
So going from mo_fullname("Staphylococcus aureus")
to "Staphylococcus aureus"
takes 0.0006 seconds - it doesn’t even start calculating if the result would be the same as the expected resulting value. That goes for all helper functions:
run_it <- microbenchmark(A = mo_species("aureus"),
B = mo_genus("Staphylococcus"),
@@ -317,14 +317,14 @@
print(run_it, unit = "ms", signif = 3)
#> Unit: milliseconds
#> expr min lq mean median uq max neval
-#> A 0.318 0.376 0.414 0.419 0.449 0.537 10
-#> B 0.343 0.397 0.437 0.447 0.479 0.522 10
-#> C 0.325 0.380 0.486 0.482 0.554 0.703 10
-#> D 0.334 0.337 0.381 0.372 0.426 0.434 10
-#> E 0.304 0.322 0.356 0.335 0.393 0.460 10
-#> F 0.295 0.323 0.370 0.362 0.424 0.463 10
-#> G 0.296 0.321 0.362 0.348 0.387 0.470 10
-#> H 0.289 0.335 0.355 0.351 0.387 0.421 10
Of course, when running mo_phylum("Firmicutes")
the function has zero knowledge about the actual microorganism, namely S. aureus. But since the result would be "Firmicutes"
too, there is no point in calculating the result. And because this package ‘knows’ all phyla of all known bacteria (according to the Catalogue of Life), it can just return the initial value immediately.
Currently supported are German, Dutch, Spanish, Italian, French and Portuguese.
diff --git a/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png b/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png index a6412a79..22db0a14 100644 Binary files a/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png and b/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png differ diff --git a/docs/articles/index.html b/docs/articles/index.html index d4d7734d..fefc0597 100644 --- a/docs/articles/index.html +++ b/docs/articles/index.html @@ -78,7 +78,7 @@ diff --git a/docs/authors.html b/docs/authors.html index 6c9d4a73..0cae75ad 100644 --- a/docs/authors.html +++ b/docs/authors.html @@ -78,7 +78,7 @@ diff --git a/docs/index.html b/docs/index.html index 9735202a..131bee7f 100644 --- a/docs/index.html +++ b/docs/index.html @@ -42,7 +42,7 @@ diff --git a/docs/news/index.html b/docs/news/index.html index 3c187f88..7bc98225 100644 --- a/docs/news/index.html +++ b/docs/news/index.html @@ -78,7 +78,7 @@ @@ -267,7 +267,7 @@ This data is updated annually - check the included version with the new functionfirst_isolate()
and eucast_rules()
, all parameters will be filled in automatically.antibiotics
data set now contains a column ears_net
.as.mo()
now knows all WHONET species abbreviations too, because more than 1,600 microbial abbreviations were added to the microorganisms.codes
data set.as.mo()
now knows all WHONET species abbreviations too, because almost 2,000 microbial abbreviations were added to the microorganisms.codes
data set.as.atc()
atc_ddd()
and atc_groups()
have been renamed atc_online_ddd()
and atc_online_groups()
. The old functions are deprecated and will be removed in a future version.guess_mo()
is now deprecated in favour of as.mo()
and will be removed in future versionsguess_atc()
is now deprecated in favour of as.atc()
and will be removed in future versionsas.mo()
:\
+as.mo()
:
UNKNOWN
. Properties of these will be translated on foreign systems in all language already previously supported: German, Dutch, French, Italian, Spanish and Portuguese:i
instead of y
and f
instead of ph
:mo_genus("qwerty", language = "es")
-# Warning:
-# one unique value (^= 100.0%) could not be coerced and is considered 'unknown': "qwerty". Use mo_failures() to review it.
-#> [1] "(género desconocido)"
# mo_fullname() uses as.mo() internally
+
+mo_fullname("Sthafilokockus aaureuz")
+#> [1] "Staphylococcus aureus"
+
+mo_fullname("S. klossi")
+#> [1] "Staphylococcus kloosii"
allow_uncertain = TRUE
is equal to uncertainty level 2. Run ?as.mo
for more info about these levels.# equal:
+as.mo(..., allow_uncertain = TRUE)
+as.mo(..., allow_uncertain = 2)
+
+# also equal:
+as.mo(..., allow_uncertain = FALSE)
+as.mo(..., allow_uncertain = 0)
as.mo(..., allow_uncertain = 3)
could lead to very unreliable results.
+UNKNOWN
. On foreign systems, properties of these will be translated to all languages already previously supported: German, Dutch, French, Italian, Spanish and Portuguese:mo_genus("qwerty", language = "es")
+# Warning:
+# one unique value (^= 100.0%) could not be coerced and is considered 'unknown': "qwerty". Use mo_failures() to review it.
+#> [1] "(género desconocido)"
as.atc()
Support for tidyverse quasiquotation! Now you can create frequency tables of function outcomes:
-# Determine genus of microorganisms (mo) in `septic_patients` data set:
-# OLD WAY
-septic_patients %>%
- mutate(genus = mo_genus(mo)) %>%
- freq(genus)
-# NEW WAY
-septic_patients %>%
- freq(mo_genus(mo))
-
-# Even supports grouping variables:
-septic_patients %>%
- group_by(gender) %>%
- freq(mo_genus(mo))
# Determine genus of microorganisms (mo) in `septic_patients` data set:
+# OLD WAY
+septic_patients %>%
+ mutate(genus = mo_genus(mo)) %>%
+ freq(genus)
+# NEW WAY
+septic_patients %>%
+ freq(mo_genus(mo))
+
+# Even supports grouping variables:
+septic_patients %>%
+ group_by(gender) %>%
+ freq(mo_genus(mo))
header
functionheader
is now set to TRUE
at default, even for markdownas.atc()
resistance_predict()
function)as.mic()
to support more values ending in (several) zeroes%like%
, it will now return the callas.atc()
as.mo
will return NAFunction as.mo
(and all mo_*
wrappers) now supports genus abbreviations with “species” attached
combine_IR
(TRUE/FALSE) to functions portion_df
and count_df
, to indicate that all values of I and R must be merged into one, so the output only consists of S vs. IR (susceptible vs. non-susceptible)portion_*(..., as_percent = TRUE)
when minimal number of isolates would not be metas.atc()
Support for grouping variables, test with:
- +Support for (un)selecting columns:
- +hms::is.hms
as.atc()
They also come with support for German, Dutch, French, Italian, Spanish and Portuguese:
-mo_gramstain("E. coli")
-# [1] "Gram negative"
-mo_gramstain("E. coli", language = "de") # German
-# [1] "Gramnegativ"
-mo_gramstain("E. coli", language = "es") # Spanish
-# [1] "Gram negativo"
-mo_fullname("S. group A", language = "pt") # Portuguese
-# [1] "Streptococcus grupo A"
mo_gramstain("E. coli")
+# [1] "Gram negative"
+mo_gramstain("E. coli", language = "de") # German
+# [1] "Gramnegativ"
+mo_gramstain("E. coli", language = "es") # Spanish
+# [1] "Gram negativo"
+mo_fullname("S. group A", language = "pt") # Portuguese
+# [1] "Streptococcus grupo A"
Furthermore, former taxonomic names will give a note about the current taxonomic name:
-mo_gramstain("Esc blattae")
-# Note: 'Escherichia blattae' (Burgess et al., 1973) was renamed 'Shimwellia blattae' (Priest and Barker, 2010)
-# [1] "Gram negative"
mo_gramstain("Esc blattae")
+# Note: 'Escherichia blattae' (Burgess et al., 1973) was renamed 'Shimwellia blattae' (Priest and Barker, 2010)
+# [1] "Gram negative"
count_R
, count_IR
, count_I
, count_SI
and count_S
to selectively count resistant or susceptible isolates
as.atc()
Functions as.mo
and is.mo
as replacements for as.bactid
and is.bactid
(since the microoganisms
data set not only contains bacteria). These last two functions are deprecated and will be removed in a future release. The as.mo
function determines microbial IDs using intelligent rules:
as.mo("E. coli")
-# [1] B_ESCHR_COL
-as.mo("MRSA")
-# [1] B_STPHY_AUR
-as.mo("S group A")
-# [1] B_STRPTC_GRA
as.mo("E. coli")
+# [1] B_ESCHR_COL
+as.mo("MRSA")
+# [1] B_STPHY_AUR
+as.mo("S group A")
+# [1] B_STRPTC_GRA
And with great speed too - on a quite regular Linux server from 2007 it takes us less than 0.02 seconds to transform 25,000 items:
- +reference_df
for as.mo
, so users can supply their own microbial IDs, name or codes as a reference tablebactid
to mo
, like:
@@ -651,12 +673,12 @@ These functions use as.atc()
antibiotics
data set: Terbinafine (D01BA02), Rifaximin (A07AA11) and Isoconazole (D01AC05)Added 163 trade names to the antibiotics
data set, it now contains 298 different trade names in total, e.g.:
first_isolate
, rows will be ignored when there’s no species availableratio
is now deprecated and will be removed in a future release, as it is not really the scope of this packageas.atc()
Support for quasiquotation in the functions series count_*
and portions_*
, and n_rsi
. This allows to check for more than 2 vectors or columns.
ggplot_rsi
and geom_rsi
so they can cope with count_df
. The new fun
parameter has value portion_df
at default, but can be set to count_df
.ggplot_rsi
when the ggplot2
package was not loadedas.atc()
Support for types (classes) list and matrix for freq
For lists, subsetting is possible:
- +Use this function to determine a valid microorganism ID (mo
). Determination is done using intelligent rules and the complete taxonomic kingdoms Archaea, Bacteria, Protozoa, Viruses and most microbial species from the kingdom Fungi (see Source), so the input can be almost anything: a full name (like "Staphylococcus aureus"
), an abbreviated name (like "S. aureus"
), an abbreviation known in the field (like "MRSA"
), or just a genus. You could also select
a genus and species column, zie Examples.
Use this function to determine a valid microorganism ID (mo
). Determination is done using intelligent rules and the complete taxonomic kingdoms Bacteria, Chromista, Protozoa, Archaea, Viruses, and most microbial species from the kingdom Fungi (see Source). The input can be almost anything: a full name (like "Staphylococcus aureus"
), an abbreviated name (like "S. aureus"
), an abbreviation known in the field (like "MRSA"
), or just a genus. Please see Examples.
a logical to indicate whether the input should be checked for less possible results, see Details
a logical (TRUE
or FALSE
) or a value between 0 and 3 to indicate whether the input should be checked for less possible results, see Details
Use the mo_property
functions to get properties based on the returned code, see Examples.
Intelligent rules
This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order:
Taxonomic kingdom: it first searches in Bacteria, then Fungi, then Protozoa
Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see section Microbial prevalence of pathogens in humans)
Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations
Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see Microbial prevalence of pathogens in humans below)
Taxonomic kingdom: it first searches in Bacteria/Chromista, then Fungi, then Protozoa, then Viruses
Breakdown of input values: from here it starts to breakdown input values to find possible matches
A couple of effects because of these rules:
Something like "stau"
or "S aur"
will return the ID of Staphylococcus aureus and not Staphylococcus auricularis
This means that looking up human pathogenic microorganisms takes less time than looking up human non-pathogenic microorganisms.
Uncertain results
-When using allow_uncertain = TRUE
(which is the default setting), it will use additional rules if all previous rules failed to get valid results. These are:
It tries to look for previously accepted (but now invalid) taxonomic names
It strips off values between brackets and the brackets itself, and re-evaluates the input with all previous rules
It strips off words from the end one by one and re-evaluates the input with all previous rules
It strips off words from the start one by one and re-evaluates the input with all previous rules
It tries to look for some manual changes which are not (yet) published to the Catalogue of Life (like Propionibacterium being Cutibacterium)
allow_uncertain = TRUE
, which is uqual to uncertainty level 2. Using allow_uncertain = FALSE
will skip all of these additional rules:(uncertainty level 1): It tries to look for only matching genera
(uncertainty level 1): It tries to look for previously accepted (but now invalid) taxonomic names
(uncertainty level 1): It tries to look for some manual changes which are not (yet) published to the Catalogue of Life (like Propionibacterium being Cutibacterium)
(uncertainty level 2): It strips off values between brackets and the brackets itself, and re-evaluates the input with all previous rules
(uncertainty level 2): It strips off words from the end one by one and re-evaluates the input with all previous rules
(uncertainty level 3): It strips off words from the start one by one and re-evaluates the input with all previous rules
(uncertainty level 3): It tries any part of the name
Examples:
You can also use e.g. as.mo(..., allow_uncertain = 1)
to only allow up to level 1 uncertainty.
Examples:
"Streptococcus group B (known as S. agalactiae)"
. The text between brackets will be removed and a warning will be thrown that the result Streptococcus group B (B_STRPT_GRB
) needs review.
"S. aureus - please mind: MRSA"
. The last word will be stripped, after which the function will try to find a match. If it does not, the second last word will be stripped, etc. Again, a warning will be thrown that the result Staphylococcus aureus (B_STPHY_AUR
) needs review.
"Fluoroquinolone-resistant Neisseria gonorrhoeae"
. The first word will be stripped, after which the function will try to find a match. A warning will be thrown that the result Neisseria gonorrhoeae (B_NESSR_GON
) needs review.
allow_uncertain = TRUE
(which is the default setting), i
Use mo_uncertainties()
to get a data.frame with all values that were coerced to a valid value, but with uncertainty.
Use mo_renamed()
to get a vector with all values that could be coerced based on an old, previously accepted taxonomic name.
Microbial prevalence of pathogens in humans
-The intelligent rules takes into account microbial prevalence of pathogens in humans. It uses three groups and every (sub)species is in the group it matches first. These groups are:
1 (most prevalent): class is Gammaproteobacteria or genus is one of: Enterococcus, Staphylococcus, Streptococcus.
2: phylum is one of: Proteobacteria, Firmicutes, Actinobacteria, Sarcomastigophora or genus is one of: Aspergillus, Bacteroides, Candida, Capnocytophaga, Chryseobacterium, Cryptococcus, Elisabethkingia, Flavobacterium, Fusobacterium, Giardia, Leptotrichia, Mycoplasma, Prevotella, Rhodotorula, Treponema, Trichophyton, Ureaplasma.
3 (least prevalent): all others.
mo_property
functions (like A data.frame
with 4,731 observations and 2 variables:
A data.frame
with 5,171 observations and 2 variables:
certe
Commonly used code of a microorganism
mo
ID of the microorganism in the microorganisms
data set