So, we can draw at least two conclusions immediately. From a data scientist perspective, the data looks clean: only values M and F. From a researcher perspective: there are slightly more men. Nothing we didn’t already know.
The data is already quite clean, but we still need to transform some variables. The bacteria column now consists of text, and we want to add more variables based on microbial IDs later on. So, we will transform this column to valid IDs. The mutate() function of the dplyr package makes this really easy:
data <- data %>%
@@ -443,10 +443,10 @@
#> Kingella kingae (no changes)
#>
#> EUCAST Expert Rules, Intrinsic Resistance and Exceptional Phenotypes (v3.1, 2016)
-#> Table 1: Intrinsic resistance in Enterobacteriaceae (1340 changes)
+#> Table 1: Intrinsic resistance in Enterobacteriaceae (1294 changes)
#> Table 2: Intrinsic resistance in non-fermentative Gram-negative bacteria (no changes)
#> Table 3: Intrinsic resistance in other Gram-negative bacteria (no changes)
-#> Table 4: Intrinsic resistance in Gram-positive bacteria (2699 changes)
+#> Table 4: Intrinsic resistance in Gram-positive bacteria (2675 changes)
#> Table 8: Interpretive rules for B-lactam agents and Gram-positive cocci (no changes)
#> Table 9: Interpretive rules for B-lactam agents and Gram-negative rods (no changes)
#> Table 10: Interpretive rules for B-lactam agents and other Gram-negative bacteria (no changes)
@@ -462,9 +462,9 @@
#> Non-EUCAST: piperacillin/tazobactam = S where piperacillin = S (no changes)
#> Non-EUCAST: trimethoprim/sulfa = S where trimethoprim = S (no changes)
#>
-#> => EUCAST rules affected 7,406 out of 20,000 rows
+#> => EUCAST rules affected 7,390 out of 20,000 rows
#> -> added 0 test results
-#> -> changed 4,039 test results (0 to S; 0 to I; 4,039 to R)So only 28.4% is suitable for resistance analysis! We can now filter on it with the filter() function, also from the dplyr package:
| isolate | @@ -654,70 +654,70 @@|||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | -2010-03-12 | -I7 | +2010-03-10 | +F7 | B_ESCHR_COL | S | S | -S | +R | S | TRUE | TRUE | |
| 2 | -2010-04-30 | -I7 | +2010-03-12 | +F7 | B_ESCHR_COL | -S | -S | -R | -S | -FALSE | -TRUE | -||
| 3 | -2010-06-28 | -I7 | -B_ESCHR_COL | -R | -S | -S | -S | -FALSE | -TRUE | -||||
| 4 | -2010-07-08 | -I7 | -B_ESCHR_COL | -S | -S | -S | -S | -FALSE | -TRUE | -||||
| 5 | -2010-08-22 | -I7 | -B_ESCHR_COL | -R | +I | I | S | S | FALSE | TRUE | |||
| 3 | +2010-06-18 | +F7 | +B_ESCHR_COL | +S | +I | +S | +S | +FALSE | +FALSE | +||||
| 4 | +2010-10-16 | +F7 | +B_ESCHR_COL | +S | +S | +S | +S | +FALSE | +FALSE | +||||
| 5 | +2010-11-12 | +F7 | +B_ESCHR_COL | +S | +S | +S | +S | +FALSE | +FALSE | +||||
| 6 | -2010-09-04 | -I7 | +2010-11-24 | +F7 | B_ESCHR_COL | -R | +S | I | S | S | @@ -726,10 +726,10 @@|||
| 7 | -2010-11-16 | -I7 | +2011-02-24 | +F7 | B_ESCHR_COL | -R | +S | S | S | S | @@ -738,47 +738,47 @@|||
| 8 | -2011-03-31 | -I7 | +2011-03-30 | +F7 | B_ESCHR_COL | -S | -S | R | S | +S | +S | TRUE | TRUE |
| 9 | -2011-05-25 | -I7 | +2011-08-09 | +F7 | B_ESCHR_COL | S | -S | R | S | +S | FALSE | -FALSE | +TRUE |
| 10 | -2011-08-25 | -I7 | +2011-08-14 | +F7 | B_ESCHR_COL | -R | S | -R | +S | +S | S | FALSE | TRUE |
Instead of 2, now 7 isolates are flagged. In total, 79.7% of all isolates are marked ‘first weighted’ - 51.3% more than when using the CLSI guideline. In real life, this novel algorithm will yield 5-10% more isolates than the classic CLSI guideline.
+Instead of 2, now 5 isolates are flagged. In total, 79.4% of all isolates are marked ‘first weighted’ - 51.1% more than when using the CLSI guideline. In real life, this novel algorithm will yield 5-10% more isolates than the classic CLSI guideline.
As with filter_first_isolate(), there’s a shortcut for this new algorithm too:
So we end up with 15,935 isolates for analysis.
+So we end up with 15,887 isolates for analysis.
We can remove unneeded columns:
@@ -786,6 +786,7 @@| date | patient_id | hospital | @@ -802,41 +803,12 @@||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010-07-18 | -Z5 | -Hospital B | -B_STRPT_PNE | -S | -S | -S | -R | -F | -Gram positive | -Streptococcus | -pneumoniae | -TRUE | -||||||||
| 2017-01-13 | -P8 | -Hospital C | -B_ESCHR_COL | -R | -S | -S | -S | -F | -Gram negative | -Escherichia | -coli | -TRUE | -||||||||
| 2014-07-31 | -K5 | +1 | +2011-05-27 | +C5 | Hospital B | B_ESCHR_COL | -S | +R | S | S | S | @@ -847,38 +819,25 @@TRUE | ||||||||
| 2017-07-12 | -U8 | -Hospital B | -B_KLBSL_PNE | -R | +2 | +2010-08-22 | +Q6 | +Hospital D | +B_ESCHR_COL | +S | S | S | S | F | Gram negative | -Klebsiella | -pneumoniae | +Escherichia | +coli | TRUE |
| 2013-09-10 | -Y1 | -Hospital B | -B_KLBSL_PNE | -R | -R | -S | -S | -F | -Gram negative | -Klebsiella | -pneumoniae | -TRUE | -||||||||
| 2017-07-30 | -X10 | +6 | +2017-05-19 | +O7 | Hospital C | B_STRPT_PNE | S | @@ -891,6 +850,54 @@pneumoniae | TRUE | |||||||||||
| 8 | +2011-01-19 | +Q7 | +Hospital B | +B_ESCHR_COL | +S | +S | +S | +S | +F | +Gram negative | +Escherichia | +coli | +TRUE | +|||||||
| 9 | +2014-06-24 | +S9 | +Hospital B | +B_STRPT_PNE | +R | +R | +S | +R | +F | +Gram positive | +Streptococcus | +pneumoniae | +TRUE | +|||||||
| 10 | +2011-10-01 | +O10 | +Hospital B | +B_STPHY_AUR | +R | +S | +R | +S | +F | +Gram positive | +Staphylococcus | +aureus | +TRUE | +
Time for the analysis!
@@ -908,9 +915,9 @@Or can be used like the dplyr way, which is easier readable:
Frequency table of genus and species from a data.frame (15,935 x 13)
Frequency table of genus and species from a data.frame (15,887 x 13)
Columns: 2
-Length: 15,935 (of which NA: 0 = 0.00%)
+Length: 15,887 (of which NA: 0 = 0.00%)
Unique: 4
Shortest: 16
Longest: 24
The functions portion_R, portion_RI, portion_I, portion_IS and portion_S can be used to determine the portion of a specific antimicrobial outcome. They can be used on their own:
Or can be used in conjuction with group_by() and summarise(), both from the dplyr package:
data_1st %>%
group_by(hospital) %>%
@@ -977,19 +984,19 @@ Longest: 24
Hospital A
-0.4740616
+0.4774125
Hospital B
-0.4748265
+0.4762754
Hospital C
-0.4910305
+0.4630957
Hospital D
-0.4777953
+0.4845070
@@ -1007,23 +1014,23 @@ Longest: 24
Hospital A
-0.4740616
-4742
+0.4774125
+4715
Hospital B
-0.4748265
-5621
+0.4762754
+5606
Hospital C
-0.4910305
-2397
+0.4630957
+2371
Hospital D
-0.4777953
-3175
+0.4845070
+3195
@@ -1043,27 +1050,27 @@ Longest: 24
Escherichia
-0.7295518
-0.9013674
-0.9772094
+0.7267089
+0.9002532
+0.9720253
Klebsiella
-0.7517505
-0.9000637
-0.9789943
+0.7233518
+0.8983364
+0.9741220
Staphylococcus
-0.7353747
-0.9168534
-0.9788399
+0.7296887
+0.9248292
+0.9787396
Streptococcus
-0.7390772
+0.7318690
0.0000000
-0.7390772
+0.7318690
diff --git a/docs/articles/AMR_files/figure-html/plot 1-1.png b/docs/articles/AMR_files/figure-html/plot 1-1.png
index 6f5f6c3b..82f75604 100644
Binary files a/docs/articles/AMR_files/figure-html/plot 1-1.png and b/docs/articles/AMR_files/figure-html/plot 1-1.png differ
diff --git a/docs/articles/AMR_files/figure-html/plot 3-1.png b/docs/articles/AMR_files/figure-html/plot 3-1.png
index 6f22a8ba..fb5561c1 100644
Binary files a/docs/articles/AMR_files/figure-html/plot 3-1.png and b/docs/articles/AMR_files/figure-html/plot 3-1.png differ
diff --git a/docs/articles/AMR_files/figure-html/plot 4-1.png b/docs/articles/AMR_files/figure-html/plot 4-1.png
index c954d5c6..d8a26ca3 100644
Binary files a/docs/articles/AMR_files/figure-html/plot 4-1.png and b/docs/articles/AMR_files/figure-html/plot 4-1.png differ
diff --git a/docs/articles/AMR_files/figure-html/plot 5-1.png b/docs/articles/AMR_files/figure-html/plot 5-1.png
index 08fe5b7b..b1b19637 100644
Binary files a/docs/articles/AMR_files/figure-html/plot 5-1.png and b/docs/articles/AMR_files/figure-html/plot 5-1.png differ
diff --git a/docs/articles/benchmarks.html b/docs/articles/benchmarks.html
index 87f36dc1..117feb83 100644
--- a/docs/articles/benchmarks.html
+++ b/docs/articles/benchmarks.html
@@ -217,14 +217,14 @@
times = 10)
print(S.aureus, unit = "ms", signif = 3)
#> Unit: milliseconds
-#> expr min lq mean median uq max neval
-#> as.mo("sau") 12.1 12.2 12.3 12.2 12.3 12.6 10
-#> as.mo("stau") 81.3 81.9 97.3 82.7 120.0 155.0 10
-#> as.mo("staaur") 12.2 12.3 12.7 12.6 13.2 13.5 10
-#> as.mo("STAAUR") 12.2 12.3 16.5 12.6 13.4 50.7 10
-#> as.mo("S. aureus") 20.1 20.1 25.2 20.1 20.2 69.9 10
-#> as.mo("S. aureus") 20.1 20.2 20.7 20.3 21.6 22.0 10
-#> as.mo("Staphylococcus aureus") 11.2 11.3 11.5 11.3 11.5 12.4 10In the table above, all measurements are shown in milliseconds (thousands of seconds). A value of 10 milliseconds means it can determine 100 input values per second. It case of 50 milliseconds, this is only 20 input values per second. The second input is the only one that has to be looked up thoroughly. All the others are known codes (the first is a WHONET code) or common laboratory codes, or common full organism names like the last one.
To achieve this speed, the as.mo function also takes into account the prevalence of human pathogenic microorganisms. The downside is of course that less prevalent microorganisms will be determined less fast. See this example for the ID of Thermus islandicus (B_THERMS_ISL), a bug probably never found before in humans:
T.islandicus <- microbenchmark(as.mo("theisl"),
@@ -235,13 +235,13 @@
times = 10)
print(T.islandicus, unit = "ms", signif = 3)
#> Unit: milliseconds
-#> expr min lq mean median uq max neval
-#> as.mo("theisl") 446.0 452.0 478.0 486.0 488.0 506 10
-#> as.mo("THEISL") 446.0 446.0 471.0 467.0 489.0 528 10
-#> as.mo("T. islandicus") 76.5 77.1 87.4 77.2 85.4 127 10
-#> as.mo("T. islandicus") 76.9 76.9 81.4 77.1 79.1 116 10
-#> as.mo("Thermus islandicus") 67.6 67.7 80.0 67.9 106.0 112 10That takes 8.5 times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like Thermus islandicus) are almost fast - these are the most probable input from most data sets.
+#> expr min lq mean median uq max neval +#> as.mo("theisl") 448.0 486.0 483.0 489.0 490.0 510.0 10 +#> as.mo("THEISL") 447.0 489.0 487.0 491.0 493.0 499.0 10 +#> as.mo("T. islandicus") 78.0 78.2 78.9 78.7 78.9 82.3 10 +#> as.mo("T. islandicus") 78.1 78.3 84.4 78.8 81.3 129.0 10 +#> as.mo("Thermus islandicus") 61.8 62.1 75.4 62.8 104.0 109.0 10 +That takes 8 times as much time on average. A value of 100 milliseconds means it can only determine ~10 different input values per second. We can conclude that looking up arbitrary codes of less prevalent microorganisms is the worst way to go, in terms of calculation performance. Full names (like Thermus islandicus) are almost fast - these are the most probable input from most data sets.
In the figure below, we compare Escherichia coli (which is very common) with Prevotella brevis (which is moderately common) and with Thermus islandicus (which is very uncommon):
par(mar = c(5, 16, 4, 2)) # set more space for left margin text (16)
@@ -287,8 +287,8 @@
print(run_it, unit = "ms", signif = 3)
#> Unit: milliseconds
#> expr min lq mean median uq max neval
-#> mo_fullname(x) 679 731 768 762 779 886 10So transforming 500,000 values (!!) of 50 unique values only takes 0.76 seconds (762 ms). You only lose time on your unique input values.
+#> mo_fullname(x) 741 746 806 778 827 968 10 +So transforming 500,000 values (!!) of 50 unique values only takes 0.78 seconds (778 ms). You only lose time on your unique input values.
So going from mo_fullname("Staphylococcus aureus") to "Staphylococcus aureus" takes 0.0004 seconds - it doesn’t even start calculating if the result would be the same as the expected resulting value. That goes for all helper functions:
So going from mo_fullname("Staphylococcus aureus") to "Staphylococcus aureus" takes 0.0006 seconds - it doesn’t even start calculating if the result would be the same as the expected resulting value. That goes for all helper functions:
run_it <- microbenchmark(A = mo_species("aureus"),
B = mo_genus("Staphylococcus"),
C = mo_fullname("Staphylococcus aureus"),
@@ -317,14 +317,14 @@
print(run_it, unit = "ms", signif = 3)
#> Unit: milliseconds
#> expr min lq mean median uq max neval
-#> A 0.366 0.446 0.563 0.523 0.681 0.785 10
-#> B 0.365 0.453 0.598 0.661 0.682 0.859 10
-#> C 0.369 0.477 0.633 0.646 0.674 1.150 10
-#> D 0.283 0.337 0.455 0.464 0.609 0.611 10
-#> E 0.309 0.333 0.422 0.393 0.527 0.606 10
-#> F 0.280 0.316 0.457 0.466 0.557 0.676 10
-#> G 0.283 0.328 0.422 0.359 0.535 0.638 10
-#> H 0.295 0.339 0.402 0.368 0.460 0.562 10Of course, when running mo_phylum("Firmicutes") the function has zero knowledge about the actual microorganism, namely S. aureus. But since the result would be "Firmicutes" too, there is no point in calculating the result. And because this package ‘knows’ all phyla of all known bacteria (according to the Catalogue of Life), it can just return the initial value immediately.
Currently supported are German, Dutch, Spanish, Italian, French and Portuguese.
diff --git a/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png b/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png index d61c4155..fabe6abc 100644 Binary files a/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png and b/docs/articles/benchmarks_files/figure-html/unnamed-chunk-5-1.png differ