@@ -283,7 +285,7 @@
Details
General info
-A microbial ID from this package (class: mo) typically looks like these examples:
+A microorganism ID from this package (class: mo) typically looks like these examples:
Code Full name
--------------- --------------------------------------
B_KLBSL Klebsiella
@@ -299,8 +301,11 @@ A microbial ID from this package (class: mo) typically looks like t
Values that cannot be coered will be considered 'unknown' and will get the MO code UNKNOWN.
Use the mo_property_* functions to get properties based on the returned code, see Examples.
-The algorithm uses data from the Catalogue of Life (see below) and from one other source (see ?microorganisms).
-Intelligent rules
+
The algorithm uses data from the Catalogue of Life (see below) and from one other source (see microorganisms).
+Self-learning algoritm
+The as.mo() function gains experience from previously determined microorganism IDs and learns from it. This drastically improves both speed and reliability. Use clear_mo_history() to reset the algorithms. Only experience from your current AMR package version is used. This is done because in the future the taxonomic tree (which is included in this package) may change for any organism and it consequently has to rebuild its knowledge.
+Usually, any guess after the first try runs 80-95% faster than the first try.
+ Intelligent rules
This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order:
Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations
Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see Microbial prevalence of pathogens in humans below)
Use mo_failures() to get a vector with all values that could not be coerced to a valid value.
Use mo_uncertainties() to get a data.frame with all values that were coerced to a valid value, but with uncertainty.
-Use mo_renamed() to get a vector with all values that could be coerced based on an old, previously accepted taxonomic name.
+Use mo_renamed() to get a data.frame with all values that could be coerced based on an old, previously accepted taxonomic name.
Microbial prevalence of pathogens in humans
The intelligent rules take into account microbial prevalence of pathogens in humans. It uses three groups and all (sub)species are in only one group. These groups are:
1 (most prevalent): class is Gammaproteobacteria or genus is one of: Enterococcus, Staphylococcus, Streptococcus.
3 (least prevalent): all others.
Group 1 contains all common Gram positives and Gram negatives, like all Enterobacteriaceae and e.g. Pseudomonas and Legionella.
-Group 2 probably contains less microbial pathogens; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.
+Group 2 contains probably less pathogenic microorganisms; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.
Source
diff --git a/docs/reference/index.html b/docs/reference/index.html
index e85c7ddc..3fa080be 100644
--- a/docs/reference/index.html
+++ b/docs/reference/index.html
@@ -78,7 +78,7 @@
AMR (for R)
- 0.7.1.9072
+ 0.7.1.9073
@@ -292,7 +292,7 @@
|
- as.mo() is.mo() mo_failures() mo_uncertainties() mo_renamed()
+ as.mo() is.mo() mo_failures() mo_uncertainties() mo_renamed() clear_mo_history()
|
Transform to microorganism ID |
@@ -384,9 +384,9 @@
|
- as.atc() p.symbol()
+ p_symbol()
|
- Deprecated functions |
+ Symbol of a p value |
|
@@ -528,7 +528,7 @@
|
|
- like() `%like%`
+ like() `%like%` `%like_case%`
|
Pattern Matching |
diff --git a/docs/reference/like.html b/docs/reference/like.html
index d313e9d8..743e75de 100644
--- a/docs/reference/like.html
+++ b/docs/reference/like.html
@@ -47,7 +47,7 @@
-
+
@@ -80,7 +80,7 @@
AMR (for R)
- 0.7.1.9067
+ 0.7.1.9073
@@ -230,13 +230,15 @@
-
Convenient wrapper around grep to match a pattern: a %like% b. It always returns a logical vector and is always case-insensitive. Also, pattern (b) can be as long as x (a) to compare items of each index in both vectors.
+
Convenient wrapper around grep to match a pattern: a %like% b. It always returns a logical vector and is always case-insensitive (use a %like_case% b for case-sensitive matching). Also, pattern (b) can be as long as x (a) to compare items of each index in both vectors, or can both have the same length to iterate over all cases.
like(x, pattern)
+ like(x, pattern, ignore.case = TRUE)
-x %like% pattern
+x %like% pattern
+
+x %like_case% pattern
Arguments
@@ -257,6 +259,11 @@
is used with a warning. Missing values are allowed except for
regexpr and gregexpr.
+
+ | ignore.case |
+ if FALSE, the pattern matching is case
+ sensitive and if TRUE, case is ignored during matching. |
+
Source
diff --git a/docs/reference/translate.html b/docs/reference/translate.html
index 14176405..252a1822 100644
--- a/docs/reference/translate.html
+++ b/docs/reference/translate.html
@@ -80,7 +80,7 @@
AMR (for R)
- 0.7.1.9072
+ 0.7.1.9073
diff --git a/inst/mo_history/mo_history.csv b/inst/mo_history/mo_history.csv
new file mode 100644
index 00000000..db967b2c
--- /dev/null
+++ b/inst/mo_history/mo_history.csv
@@ -0,0 +1 @@
+"","x","mo","uncertainty_level","package_v"
diff --git a/man/as.mo.Rd b/man/as.mo.Rd
index f28bf0ae..62f40090 100644
--- a/man/as.mo.Rd
+++ b/man/as.mo.Rd
@@ -1,5 +1,5 @@
% Generated by roxygen2: do not edit by hand
-% Please edit documentation in R/mo.R
+% Please edit documentation in R/mo.R, R/mo_history.R
\name{as.mo}
\alias{as.mo}
\alias{mo}
@@ -7,6 +7,7 @@
\alias{mo_failures}
\alias{mo_uncertainties}
\alias{mo_renamed}
+\alias{clear_mo_history}
\title{Transform to microorganism ID}
\usage{
as.mo(x, Becker = FALSE, Lancefield = FALSE, allow_uncertain = TRUE,
@@ -19,6 +20,8 @@ mo_failures()
mo_uncertainties()
mo_renamed()
+
+clear_mo_history(...)
}
\arguments{
\item{x}{a character vector or a \code{data.frame} with one or two columns}
@@ -45,7 +48,7 @@ Use this function to determine a valid microorganism ID (\code{mo}). Determinati
}
\details{
\strong{General info} \cr
-A microbial ID from this package (class: \code{mo}) typically looks like these examples:\cr
+A microorganism ID from this package (class: \code{mo}) typically looks like these examples:\cr
\preformatted{
Code Full name
--------------- --------------------------------------
@@ -65,7 +68,13 @@ Values that cannot be coered will be considered 'unknown' and will get the MO co
Use the \code{\link{mo_property}_*} functions to get properties based on the returned code, see Examples.
-The algorithm uses data from the Catalogue of Life (see below) and from one other source (see \code{?microorganisms}).
+The algorithm uses data from the Catalogue of Life (see below) and from one other source (see \code{\link{microorganisms}}).
+
+\strong{Self-learning algoritm} \cr
+The \code{as.mo()} function gains experience from previously determined microorganism IDs and learns from it. This drastically improves both speed and reliability. Use \code{clear_mo_history()} to reset the algorithms. Only experience from your current \code{AMR} package version is used. This is done because in the future the taxonomic tree (which is included in this package) may change for any organism and it consequently has to rebuild its knowledge.
+
+Usually, any guess after the first try runs 80-95\% faster than the first try.
+
\strong{Intelligent rules} \cr
This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order:
@@ -105,7 +114,7 @@ Use \code{mo_failures()} to get a vector with all values that could not be coerc
Use \code{mo_uncertainties()} to get a data.frame with all values that were coerced to a valid value, but with uncertainty.
-Use \code{mo_renamed()} to get a vector with all values that could be coerced based on an old, previously accepted taxonomic name.
+Use \code{mo_renamed()} to get a data.frame with all values that could be coerced based on an old, previously accepted taxonomic name.
\strong{Microbial prevalence of pathogens in humans} \cr
The intelligent rules take into account microbial prevalence of pathogens in humans. It uses three groups and all (sub)species are in only one group. These groups are:
@@ -117,7 +126,7 @@ The intelligent rules take into account microbial prevalence of pathogens in hum
Group 1 contains all common Gram positives and Gram negatives, like all Enterobacteriaceae and e.g. \emph{Pseudomonas} and \emph{Legionella}.
-Group 2 probably contains less microbial pathogens; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.
+Group 2 contains probably less pathogenic microorganisms; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.
}
\section{Source}{
diff --git a/man/like.Rd b/man/like.Rd
index 88ec5ae4..4d4169ff 100755
--- a/man/like.Rd
+++ b/man/like.Rd
@@ -3,14 +3,17 @@
\name{like}
\alias{like}
\alias{\%like\%}
+\alias{\%like_case\%}
\title{Pattern Matching}
\source{
Idea from the \href{https://github.com/Rdatatable/data.table/blob/master/R/like.R}{\code{like} function from the \code{data.table} package}, but made it case insensitive at default and let it support multiple patterns. Also, if the regex fails the first time, it tries again with \code{perl = TRUE}.
}
\usage{
-like(x, pattern)
+like(x, pattern, ignore.case = TRUE)
x \%like\% pattern
+
+x \%like_case\% pattern
}
\arguments{
\item{x}{a character vector where matches are sought, or an
@@ -24,12 +27,15 @@ x \%like\% pattern
character vector of length 2 or more is supplied, the first element
is used with a warning. Missing values are allowed except for
\code{regexpr} and \code{gregexpr}.}
+
+\item{ignore.case}{if \code{FALSE}, the pattern matching is \emph{case
+ sensitive} and if \code{TRUE}, case is ignored during matching.}
}
\value{
A \code{logical} vector
}
\description{
-Convenient wrapper around \code{\link[base]{grep}} to match a pattern: \code{a \%like\% b}. It always returns a \code{logical} vector and is always case-insensitive. Also, \code{pattern} (\code{b}) can be as long as \code{x} (\code{a}) to compare items of each index in both vectors.
+Convenient wrapper around \code{\link[base]{grep}} to match a pattern: \code{a \%like\% b}. It always returns a \code{logical} vector and is always case-insensitive (use \code{a \%like_case\% b} for case-sensitive matching). Also, \code{pattern} (\code{b}) can be as long as \code{x} (\code{a}) to compare items of each index in both vectors, or can both have the same length to iterate over all cases.
}
\details{
Using RStudio? This function can also be inserted from the Addins menu and can have its own Keyboard Shortcut like Ctrl+Shift+L or Cmd+Shift+L (see Tools > Modify Keyboard Shortcuts...).
diff --git a/tests/testthat/test-mo.R b/tests/testthat/test-mo.R
index c3646ea1..9fbaa3a6 100644
--- a/tests/testthat/test-mo.R
+++ b/tests/testthat/test-mo.R
@@ -22,6 +22,8 @@
context("mo.R")
test_that("as.mo works", {
+
+ clear_mo_history(force = TRUE)
library(dplyr)
MOs <- AMR::microorganisms %>% filter(!is.na(mo), nchar(mo) > 3)
diff --git a/vignettes/benchmarks.Rmd b/vignettes/benchmarks.Rmd
index 8aaf8d10..a5928899 100755
--- a/vignettes/benchmarks.Rmd
+++ b/vignettes/benchmarks.Rmd
@@ -20,7 +20,7 @@ knitr::opts_chunk$set(
comment = "#",
fig.width = 7.5,
fig.height = 4.5,
- dpi = 150
+ dpi = 75
)
```
@@ -110,26 +110,40 @@ That takes `r round(mean(T.islandicus$time, na.rm = TRUE) / mean(S.aureus$time,
In the figure below, we compare *Escherichia coli* (which is very common) with *Prevotella brevis* (which is moderately common) and with *Thermus islandicus* (which is uncommon):
```{r, echo = FALSE}
-ggplot.bm(
- microbenchmark(as.mo("Escherichia coli"),
- as.mo("E. coli"),
- times = 10), title = "Very common")
+# ggplot.bm(
+# microbenchmark(as.mo("Escherichia coli"),
+# as.mo("E. coli"),
+# times = 10), title = "Very common")
+#
+# ggplot.bm(
+# microbenchmark(as.mo("Prevotella brevis"),
+# as.mo("P. brevis"),
+# times = 10), title = "Moderately common")
+#
+# ggplot.bm(
+# microbenchmark(as.mo("Thermus islandicus"),
+# as.mo("T. islandicus"),
+# times = 10), title = "Uncommon")
-ggplot.bm(
- microbenchmark(as.mo("Prevotella brevis"),
- as.mo("P. brevis"),
- times = 10), title = "Moderately common")
-
-ggplot.bm(
- microbenchmark(as.mo("Thermus islandicus"),
- as.mo("T. islandicus"),
- times = 10), title = "Uncommon")
+par(mar = c(5, 16, 4, 2))
+boxplot(microbenchmark(
+ 'as.mo("Thermus islandicus")' = as.mo("Thermus islandicus"),
+ 'as.mo("Prevotella brevis")' = as.mo("Prevotella brevis"),
+ 'as.mo("Escherichia coli")' = as.mo("Escherichia coli"),
+ 'as.mo("T. islandicus")' = as.mo("T. islandicus"),
+ 'as.mo("P. brevis")' = as.mo("P. brevis"),
+ 'as.mo("E. coli")' = as.mo("E. coli"),
+ times = 10),
+ horizontal = TRUE, las = 1, unit = "s", log = FALSE,
+ xlab = "", ylab = "Time in seconds", ylim = c(0, 0.5),
+ main = "Benchmarks per prevalence")
```
-```{r, echo = FALSE, eval = FALSE}
-# In reality, the `as.mo()` functions **learns from its own output to speed up determinations for next times**. In above figure, this effect was disabled to show the difference with the boxplot below - when you would use `as.mo()` yourself:
+In reality, the `as.mo()` functions **learns from its own output to speed up determinations for next times**. In above figure, this effect was disabled to show the difference with the boxplot below - when you would use `as.mo()` yourself:
-clean_mo_history()
+```{r, echo = FALSE}
+
+clear_mo_history()
par(mar = c(5, 16, 4, 2))
boxplot(microbenchmark(
'as.mo("Thermus islandicus")' = as.mo("Thermus islandicus", force_mo_history = TRUE),
@@ -142,10 +156,10 @@ boxplot(microbenchmark(
horizontal = TRUE, las = 1, unit = "s", log = FALSE,
xlab = "", ylab = "Time in seconds", ylim = c(0, 0.5),
main = "Benchmarks per prevalence")
-
-# The highest outliers are the first times. All next determinations were done in only thousands of seconds. For now, learning only works per session. If R is closed or terminated, the algorithms reset. This will probably be resolved in a next version.
```
+The highest outliers are the first times. All next determinations were done in only thousands of seconds.
+
Uncommon microorganisms take a lot more time than common microorganisms. To relieve this pitfall and further improve performance, two important calculations take almost no time at all: **repetitive results** and **already precalculated results**.
### Repetitive results