#' A data set containing old (previously valid or accepted) taxonomic names according to the Catalogue of Life. This data set is used internally by \code{\link{as.mo}}.
#' @inheritSection catalogue_of_life Catalogue of Life
#' @format A \code{\link{data.frame}} with 24,246 observations and 4 variables:
#' @format A \code{\link{data.frame}} with 24,246 observations and 5 variables:
#' \describe{
#' \item{\code{col_id}}{Catalogue of Life ID that was originally given}
#' \item{\code{col_id_new}}{New Catalogue of Life ID that responds to an entry in the \code{\link{microorganisms}} data set}
#' \item{\code{fullname}}{Old full taxonomic name of the microorganism}
#' \item{\code{ref}}{Author(s) and year of concerning scientific publication}
#' \item{\code{prevalence}}{Prevalence of the microorganism, see \code{?as.mo}}
#' }
#' @source Catalogue of Life: Annual Checklist (public online taxonomic database), \url{http://www.catalogueoflife.org} (check included annual version with \code{\link{catalogue_of_life_version}()}).
#' \item{Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations}
#' \item{Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see \emph{Microbial prevalence of pathogens in humans} below)}
#' \item{Taxonomic kingdom: it first searches in Bacteria/Chromista, then Fungi, then Protozoa}
#' \item{Taxonomic kingdom: it first searches in Bacteria, then Fungi, then Protozoa, then Archaea, then others}
#' \item{Breakdown of input values: from here it starts to breakdown input values to find possible matches}
#' }
#'
#'
#' A couple of effects because of these rules:
#' \itemize{
#' \item{\code{"E. coli"} will return the ID of \emph{Escherichia coli} and not \emph{Entamoeba coli}, although the latter would alphabetically come first}
<spanclass="version label label-default"data-toggle="tooltip"data-placement="bottom"title="Latest development version">0.7.1.9076</span>
<spanclass="version label label-default"data-toggle="tooltip"data-placement="bottom"title="Latest development version">0.7.1.9077</span>
</span>
</div>
@@ -199,7 +199,7 @@
<p>Using the <code>microbenchmark</code> package, we can review the calculation performance of this function. Its function <code><ahref="https://www.rdocumentation.org/packages/microbenchmark/topics/microbenchmark">microbenchmark()</a></code> runs different input expressions independently of each other and measures their time-to-result.</p>
<p>In the next test, we try to ‘coerce’ different input values for <em>Staphylococcus aureus</em>. The actual result is the same every time: it returns its MO code <code>B_STPHY_AURS</code> (<em>B</em> stands for <em>Bacteria</em>, the taxonomic kingdom).</p>
<p>In the next test, we try to ‘coerce’ different input values for <em>Staphylococcus aureus</em>. The actual result is the same every time: it returns its microorganism code <code>B_STPHY_AURS</code> (<em>B</em> stands for <em>Bacteria</em>, the taxonomic kingdom).</p>
<p>In the table above, all measurements are shown in milliseconds (thousands of seconds). A value of 5 milliseconds means it can determine 200 input values per second. It case of 100 milliseconds, this is only 10 input values per second. The second input is the only one that has to be looked up thoroughly. All the others are known codes (the first one is a WHONET code) or common laboratory codes, or common full organism names like the last one. Full organism names are always preferred.</p>
<p>To achieve this speed, the <code>as.mo</code> function also takes into account the prevalence of human pathogenic microorganisms. The downside is of course that less prevalent microorganisms will be determined less fast. See this example for the ID of <em>Thermus islandicus</em> (<code>B_THERMS_ISLN</code>), a bug probably never found before in humans:</p>
<p>To achieve this speed, the <code>as.mo</code> function also takes into account the prevalence of human pathogenic microorganisms. The downside is of course that less prevalent microorganisms will be determined less fast. See this example for the ID of <em>Methanosarcina semesiae</em> (<code>B_MTHNSR_SEMS</code>), a bug probably never found before in humans:</p>
<p>That takes 8.2 times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like <em>Thermus islandicus</em>) are almost fast - these are the most probable input from most data sets.</p>
<p>In the figure below, we compare <em>Escherichia coli</em> (which is very common) with <em>Prevotella brevis</em> (which is moderately common) and with <em>Thermus islandicus</em> (which is uncommon):</p>
<pre><code># Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.</code></pre>
<p>That takes 15.7 times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like <em>Methanosarcina semesiae</em>) are almost fast - these are the most probable input from most data sets.</p>
<p>In the figure below, we compare <em>Escherichia coli</em> (which is very common) with <em>Prevotella brevis</em> (which is moderately common) and with <em>Methanosarcina semesiae</em> (which is uncommon):</p>
<p>In reality, the <code><ahref="../reference/as.mo.html">as.mo()</a></code> functions <strong>learns from its own output to speed up determinations for next times</strong>. In above figure, this effect was disabled to show the difference with the boxplot below - when you would use <code><ahref="../reference/as.mo.html">as.mo()</a></code> yourself:</p>
<pre><code># NOTE: results are saved to /Users/msberends/Library/R/3.6/library/AMR/mo_history/mo_history.csv.
# Warning:
# Result of one value was guessed with uncertainty. Use mo_uncertainties() to review it.</code></pre>
<p>The highest outliers are the first times. All next determinations were done in only thousands of seconds.</p>
<p>Uncommon microorganisms take a lot more time than common microorganisms. To relieve this pitfall and further improve performance, two important calculations take almost no time at all: <strong>repetitive results</strong> and <strong>already precalculated results</strong>.</p>
<p>Repetitive results are unique values that are present more than once. Unique values will only be calculated once by <code><ahref="../reference/as.mo.html">as.mo()</a></code>. We will use <code><ahref="../reference/mo_property.html">mo_name()</a></code> for this test - a helper function that returns the full microbial name (genus, species and possibly subspecies) which uses <code><ahref="../reference/as.mo.html">as.mo()</a></code> internally.</p>
<p>What about precalculated results? If the input is an already precalculated result of a helper function like <code><ahref="../reference/mo_property.html">mo_name()</a></code>, it almost doesn’t take any time at all (see ‘C’ below):</p>
<aclass="sourceLine"id="cb5-7"data-line-number="7"><spanclass="co"># expr min lq mean median uq max neval</span></a>
<aclass="sourceLine"id="cb5-8"data-line-number="8"><spanclass="co"># A 6.280 6.560 9.940 6.720 6.860 39.30 10</span></a>
<aclass="sourceLine"id="cb5-9"data-line-number="9"><spanclass="co"># B 22.500 22.900 24.300 23.000 24.900 30.90 10</span></a>
<aclass="sourceLine"id="cb5-10"data-line-number="10"><spanclass="co"># C 0.805 0.829 0.871 0.847 0.869 1.09 10</span></a></code></pre></div>
<p>So going from <code><ahref="../reference/mo_property.html">mo_name("Staphylococcus aureus")</a></code> to <code>"Staphylococcus aureus"</code> takes 0.0008 seconds - it doesn’t even start calculating <em>if the result would be the same as the expected resulting value</em>. That goes for all helper functions:</p>
<aclass="sourceLine"id="cb6-12"data-line-number="12"><spanclass="co"># expr min lq mean median uq max neval</span></a>
<aclass="sourceLine"id="cb6-13"data-line-number="13"><spanclass="co"># A 0.456 0.457 0.472 0.465 0.493 0.498 10</span></a>
<aclass="sourceLine"id="cb6-14"data-line-number="14"><spanclass="co"># B 0.629 0.640 0.713 0.668 0.752 0.956 10</span></a>
<aclass="sourceLine"id="cb6-15"data-line-number="15"><spanclass="co"># C 0.798 0.811 0.840 0.832 0.840 0.965 10</span></a>
<aclass="sourceLine"id="cb6-16"data-line-number="16"><spanclass="co"># D 0.428 0.453 0.473 0.464 0.503 0.518 10</span></a>
<aclass="sourceLine"id="cb6-17"data-line-number="17"><spanclass="co"># E 0.446 0.477 0.513 0.495 0.525 0.648 10</span></a>
<aclass="sourceLine"id="cb6-18"data-line-number="18"><spanclass="co"># F 0.466 0.473 0.496 0.484 0.521 0.545 10</span></a>
<aclass="sourceLine"id="cb6-19"data-line-number="19"><spanclass="co"># G 0.457 0.461 0.477 0.468 0.486 0.545 10</span></a>
<aclass="sourceLine"id="cb6-20"data-line-number="20"><spanclass="co"># H 0.456 0.467 0.478 0.477 0.482 0.512 10</span></a></code></pre></div>
<p>Of course, when running <code><ahref="../reference/mo_property.html">mo_phylum("Firmicutes")</a></code> the function has zero knowledge about the actual microorganism, namely <em>S. aureus</em>. But since the result would be <code>"Firmicutes"</code> too, there is no point in calculating the result. And because this package ‘knows’ all phyla of all known bacteria (according to the Catalogue of Life), it can just return the initial value immediately.</p>
<ahref="#results-in-other-languages"class="anchor"></a>Results in other languages</h3>
<p>When the system language is non-English and supported by this <code>AMR</code> package, some functions will have a translated result. This almost does’t take extra time:</p>
<divclass="sourceCode"id="cb9"><preclass="sourceCode r"><codeclass="sourceCode r"><aclass="sourceLine"id="cb9-1"data-line-number="1"><spanclass="kw"><ahref="../reference/mo_property.html">mo_name</a></span>(<spanclass="st">"CoNS"</span>, <spanclass="dt">language =</span><spanclass="st">"en"</span>) <spanclass="co"># or just mo_name("CoNS") on an English system</span></a>
<aclass="sourceLine"id="cb9-4"data-line-number="4"><spanclass="kw"><ahref="../reference/mo_property.html">mo_name</a></span>(<spanclass="st">"CoNS"</span>, <spanclass="dt">language =</span><spanclass="st">"es"</span>) <spanclass="co"># or just mo_name("CoNS") on a Spanish system</span></a>
<aclass="sourceLine"id="cb9-7"data-line-number="7"><spanclass="kw"><ahref="../reference/mo_property.html">mo_name</a></span>(<spanclass="st">"CoNS"</span>, <spanclass="dt">language =</span><spanclass="st">"nl"</span>) <spanclass="co"># or just mo_name("CoNS") on a Dutch system</span></a>
<divclass="sourceCode"id="cb7"><preclass="sourceCode r"><codeclass="sourceCode r"><aclass="sourceLine"id="cb7-1"data-line-number="1"><spanclass="kw"><ahref="../reference/mo_property.html">mo_name</a></span>(<spanclass="st">"CoNS"</span>, <spanclass="dt">language =</span><spanclass="st">"en"</span>) <spanclass="co"># or just mo_name("CoNS") on an English system</span></a>
<aclass="sourceLine"id="cb7-4"data-line-number="4"><spanclass="kw"><ahref="../reference/mo_property.html">mo_name</a></span>(<spanclass="st">"CoNS"</span>, <spanclass="dt">language =</span><spanclass="st">"es"</span>) <spanclass="co"># or just mo_name("CoNS") on a Spanish system</span></a>
<aclass="sourceLine"id="cb7-7"data-line-number="7"><spanclass="kw"><ahref="../reference/mo_property.html">mo_name</a></span>(<spanclass="st">"CoNS"</span>, <spanclass="dt">language =</span><spanclass="st">"nl"</span>) <spanclass="co"># or just mo_name("CoNS") on a Dutch system</span></a>
<spanclass="version label label-default"data-toggle="tooltip"data-placement="bottom"title="Latest development version">0.7.1.9076</span>
<spanclass="version label label-default"data-toggle="tooltip"data-placement="bottom"title="Latest development version">0.7.1.9077</span>
</span>
</div>
@@ -309,7 +309,7 @@ The <code>as.mo()</code> function gains experience from previously determined mi
This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order:</p><ul>
<li><p>Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations</p></li>
<li><p>Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see <em>Microbial prevalence of pathogens in humans</em> below)</p></li>
<li><p>Taxonomic kingdom: it first searches in Bacteria/Chromista, then Fungi, then Protozoa</p></li>
<li><p>Taxonomic kingdom: it first searches in Bacteria, then Fungi, then Protozoa, then Archaea, then others</p></li>
<li><p>Breakdown of input values: from here it starts to breakdown input values to find possible matches</p></li>
</ul>
<p>A couple of effects because of these rules:</p><ul>
<p>A <code><ahref='https://www.rdocumentation.org/packages/base/topics/data.frame'>data.frame</a></code> with 24,246 observations and 4 variables:</p><dlclass='dl-horizontal'>
<p>A <code><ahref='https://www.rdocumentation.org/packages/base/topics/data.frame'>data.frame</a></code> with 24,246 observations and 5 variables:</p><dlclass='dl-horizontal'>
<dt><code>col_id</code></dt><dd><p>Catalogue of Life ID that was originally given</p></dd>
<dt><code>col_id_new</code></dt><dd><p>New Catalogue of Life ID that responds to an entry in the <code><ahref='microorganisms.html'>microorganisms</a></code> data set</p></dd>
<dt><code>fullname</code></dt><dd><p>Old full taxonomic name of the microorganism</p></dd>
<dt><code>ref</code></dt><dd><p>Author(s) and year of concerning scientific publication</p></dd>
<dt><code>prevalence</code></dt><dd><p>Prevalence of the microorganism, see <code><ahref='as.mo.html'>?as.mo</a></code></p></dd>
@@ -81,10 +81,11 @@ This function uses intelligent rules to help getting fast and logical results. I
\itemize{
\item{Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations}
\item{Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see \emph{Microbial prevalence of pathogens in humans} below)}
\item{Taxonomic kingdom: it first searches in Bacteria/Chromista, then Fungi, then Protozoa}
\item{Taxonomic kingdom: it first searches in Bacteria, then Fungi, then Protozoa, then Archaea, then others}
\item{Breakdown of input values: from here it starts to breakdown input values to find possible matches}
}
A couple of effects because of these rules:
\itemize{
\item{\code{"E. coli"} will return the ID of \emph{Escherichia coli} and not \emph{Entamoeba coli}, although the latter would alphabetically come first}
\title{Data set with previously accepted taxonomic names}
\format{A \code{\link{data.frame}} with 24,246 observations and 4 variables:
\format{A \code{\link{data.frame}} with 24,246 observations and 5 variables:
\describe{
\item{\code{col_id}}{Catalogue of Life ID that was originally given}
\item{\code{col_id_new}}{New Catalogue of Life ID that responds to an entry in the \code{\link{microorganisms}} data set}
\item{\code{fullname}}{Old full taxonomic name of the microorganism}
\item{\code{ref}}{Author(s) and year of concerning scientific publication}
\item{\code{prevalence}}{Prevalence of the microorganism, see \code{?as.mo}}
}}
\source{
Catalogue of Life: Annual Checklist (public online taxonomic database), \url{http://www.catalogueoflife.org} (check included annual version with \code{\link{catalogue_of_life_version}()}).
In the next test, we try to 'coerce' different input values for *Staphylococcus aureus*. The actual result is the same every time: it returns its MO code `B_STPHY_AURS` (*B* stands for *Bacteria*, the taxonomic kingdom).
In the next test, we try to 'coerce' different input values for *Staphylococcus aureus*. The actual result is the same every time: it returns its microorganism code `B_STPHY_AURS` (*B* stands for *Bacteria*, the taxonomic kingdom).
But the calculation time differs a lot:
```{r, echo = FALSE}
clear_mo_history()
```
```{r, warning=FALSE}
S.aureus <- microbenchmark(
as.mo("sau"), # WHONET code
@@ -93,68 +96,61 @@ ggplot.bm(S.aureus)
In the table above, all measurements are shown in milliseconds (thousands of seconds). A value of 5 milliseconds means it can determine 200 input values per second. It case of 100 milliseconds, this is only 10 input values per second. The second input is the only one that has to be looked up thoroughly. All the others are known codes (the first one is a WHONET code) or common laboratory codes, or common full organism names like the last one. Full organism names are always preferred.
To achieve this speed, the `as.mo` function also takes into account the prevalence of human pathogenic microorganisms. The downside is of course that less prevalent microorganisms will be determined less fast. See this example for the ID of *Thermus islandicus* (`B_THERMS_ISLN`), a bug probably never found before in humans:
```{r, warning=FALSE}
T.islandicus <- microbenchmark(as.mo("theisl"),
as.mo("THEISL"),
as.mo("T. islandicus"),
as.mo("T. islandicus"),
as.mo("Thermus islandicus"),
times = 10)
print(T.islandicus, unit = "ms", signif = 2)
```
That takes `r round(mean(T.islandicus$time, na.rm = TRUE) / mean(S.aureus$time, na.rm = TRUE), 1)` times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like *Thermus islandicus*) are almost fast - these are the most probable input from most data sets.
In the figure below, we compare *Escherichia coli* (which is very common) with *Prevotella brevis* (which is moderately common) and with *Thermus islandicus* (which is uncommon):
To achieve this speed, the `as.mo` function also takes into account the prevalence of human pathogenic microorganisms. The downside is of course that less prevalent microorganisms will be determined less fast. See this example for the ID of *Methanosarcina semesiae* (`B_MTHNSR_SEMS`), a bug probably never found before in humans:
```{r, echo = FALSE}
# ggplot.bm(
# microbenchmark(as.mo("Escherichia coli"),
# as.mo("E. coli"),
# times = 10), title = "Very common")
#
# ggplot.bm(
# microbenchmark(as.mo("Prevotella brevis"),
# as.mo("P. brevis"),
# times = 10), title = "Moderately common")
#
# ggplot.bm(
# microbenchmark(as.mo("Thermus islandicus"),
# as.mo("T. islandicus"),
# times = 10), title = "Uncommon")
clear_mo_history()
```
```{r, warning=FALSE}
M.semesiae <- microbenchmark(as.mo("metsem"),
as.mo("METSEM"),
as.mo("M. semesiae"),
as.mo("M. semesiae"),
as.mo("Methanosarcina semesiae"),
times = 10)
print(M.semesiae, unit = "ms", signif = 4)
```
That takes `r round(mean(M.semesiae$time, na.rm = TRUE) / mean(S.aureus$time, na.rm = TRUE), 1)` times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like *Methanosarcina semesiae*) are almost fast - these are the most probable input from most data sets.
In the figure below, we compare *Escherichia coli* (which is very common) with *Prevotella brevis* (which is moderately common) and with *Methanosarcina semesiae* (which is uncommon):
horizontal = TRUE, las = 1, unit = "s", log = TRUE,
xlab = "", ylab = "Time in seconds (log)",
main = "Benchmarks per prevalence")
```
In reality, the `as.mo()` functions **learns from its own output to speed up determinations for next times**. In above figure, this effect was disabled to show the difference with the boxplot below - when you would use `as.mo()` yourself:
@@ -197,6 +193,9 @@ So transforming 500,000 values (!!) of `r n_distinct(x)` unique values only take
What about precalculated results? If the input is an already precalculated result of a helper function like `mo_name()`, it almost doesn't take any time at all (see 'C' below):
So going from `mo_name("Staphylococcus aureus")` to `"Staphylococcus aureus"` takes `r format(round(run_it %>% filter(expr == "C") %>% pull(time) %>% median() / 1e9, 4), scientific = FALSE)` seconds - it doesn't even start calculating *if the result would be the same as the expected resulting value*. That goes for all helper functions:
@@ -226,6 +228,9 @@ Of course, when running `mo_phylum("Firmicutes")` the function has zero knowledg
When the system language is non-English and supported by this `AMR` package, some functions will have a translated result. This almost does't take extra time:
```{r, echo = FALSE}
clear_mo_history()
```
```{r}
mo_name("CoNS", language = "en") # or just mo_name("CoNS") on an English system
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
