### Install EndoMineR from a local zip file

Source: https://docs.ropensci.org/EndoMineR/index.html

Manual installation process for users without GitHub access, involving setting the working directory, unzipping, building the package, and installing the resulting tarball.

```R
setwd("C:/Users/Desktop/")

#On windows you cand cd to change the directory or us pushd to create a temporary directory indtead of cd and then setwd to the temporary directory
unzip("EndoMineR.zip")
file.rename("EndoMineR.zip-master", "EndoMineR.zip")
shell("R CMD build EndoMineR.zip")

#Then install the resulting tarball with:

install.packages("EndoMineR_0.2.0.9000.tar.gz", repos = NULL)
```

--------------------------------

### EventList example: unlist all events

Source: https://docs.ropensci.org/EndoMineR/reference/EventList.html

Example showing how to unlist the EventList output to get a vector of all standardised events. The code is commented out in the source.

```r
# unique(unlist(EventList(), use.names = FALSE))
```

--------------------------------

### scale_fill_Publication example

Source: https://docs.ropensci.org/EndoMineR/reference/scale_fill_Publication.html

The documentation states that no example is needed for this function.

```R
# None needed
```

--------------------------------

### EntityPairs_OneSentence Example

Source: https://docs.ropensci.org/EndoMineR/reference/EntityPairs_OneSentence.html

An example showing how to call the function with histology and location lists.

```R
# tbb<-EntityPairs_OneSentence(Mypath$Histology,HistolType(),LocationList())
```

--------------------------------

### Example usage of Barretts_FUType with Mypath data

Source: https://docs.ropensci.org/EndoMineR/reference/Barretts_FUType.html

Example workflow for Barretts_FUType. It starts with the Mypath demo dataset, cleans histology columns, merges with endoscopy data, runs Barretts_PathStage and Barretts_PragueScore, then applies Barretts_FUType. The example shows expected warnings about coercion and NAs.

```r
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
v <- Mypath
v$NumBx <- HistolNumbOfBx(v$Macroscopicdescription, "specimen")
v$BxSize <- HistolBxSize(v$Macroscopicdescription)
# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", v, "Dateofprocedure",
  "HospitalNumber"
)
# The function relies on the other Barrett's functions being run as well:
v$IMorNoIM <- Barretts_PathStage(v, "Histology")
v <- Barretts_PragueScore(v, "Findings")
#> Warning: argument is not an atomic vector; coercing

# The follow-up group depends on the histology and the Prague score for a
# patient so it takes the processed Barrett's data and then looks in the
# Findings column for permutations of the Prague score.
v$FU_Type <- Barretts_FUType(v, "CStage", "MStage", "IMorNoIM")
#> Warning: There were 6 warnings in `mutate()`.
#> The first warning was:
#> ℹ In argument: `FU_Type = case_when(...)`.
#> Caused by warning:
#> ! NAs introduced by coercion
#> ℹ Run `dplyr::last_dplyr_warnings()` to see the 5 remaining warnings.
rm(v)
```

--------------------------------

### Example of spellCheck preparation

Source: https://docs.ropensci.org/EndoMineR/reference/spellCheck.html

An example showing how to prepare data for spell checking by splitting and converting strings to lowercase.

```R
L <- tolower(stringr::str_split(HistolType(),"\\|"))
```

--------------------------------

### Install EndoMineR from GitHub

Source: https://docs.ropensci.org/EndoMineR/index.html

Use devtools to install the latest version of the package directly from the ropenSci repository.

```R
# install.packages("devtools")
devtools::install_github("ropenSci/EndoMineR")
```

--------------------------------

### MyImgLibrary Example

Source: https://docs.ropensci.org/EndoMineR/reference/MyImgLibrary.html

Example call to MyImgLibrary specifying the HTML file path, the delimiter phrase, and the image folder location.

```R
# MyImgLibrary("~/Images Captured with Proc Data Audit_Findings1.html",
#                         "procedureperformed","~/")
```

--------------------------------

### Example: Extract Prague score from demo dataset

Source: https://docs.ropensci.org/EndoMineR/reference/Eosinophilics.html

Workflow example that extrapolates relevant columns from the Mypath demo dataset, merges histology with endoscopy data, and applies Eosinophilics. The example uses HistolNumbOfBx, HistolBxSize, and Endomerge2 functions.

```r
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
v <- Mypath
v$NumBx <- HistolNumbOfBx(v$Macroscopicdescription, "specimen")
v$BxSize <- HistolBxSize(v$Macroscopicdescription)
# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", v, "Dateofprocedure",
  "HospitalNumber"
)

aa <- Eosinophilics(v, "Findings", "Histology", "Indications")
```

--------------------------------

### IBD_Scores example placeholder

Source: https://docs.ropensci.org/EndoMineR/reference/IBD_Scores.html

Placeholder example for the IBD_Scores function. The actual example is not yet provided in the documentation.

```r
 # Example to be provided
```

--------------------------------

### HistolBxSize example

Source: https://docs.ropensci.org/EndoMineR/reference/HistolBxSize.html

Example call to HistolBxSize using the Macroscopicdescription column from a data frame named Mypath.

```r
rr <- HistolBxSize(Mypath$Macroscopicdescription)
```

--------------------------------

### Example usage of sanity

Source: https://docs.ropensci.org/EndoMineR/reference/sanity.html

An example showing how to provide a path to an R script to generate a consort diagram.

```R
#pathName<-paste0(here::here(),"/inst/TemplateProject/munge/PreProcessing.R")
#sanity(pathName)
# This creates a consort diagram from any R script (not Rmd). It
# basically tells you how all the dataframes are related and how many
# rows each dataframe has so you can see if any data has been lost
# on the way.
```

--------------------------------

### PatientFlow_CircosPlots Example

Source: https://docs.ropensci.org/EndoMineR/reference/PatientFlow_CircosPlots.html

An example demonstrating how to prepare data and invoke the PatientFlow_CircosPlots function to visualize patient flow between procedures.

```R
# This function builds a circos plot which gives a more aggregated
# overview of how patients flow from one state to another than the
# SurveySankey function
# Build a list of procedures
Event <- list(
  x1 = "Therapeutic- Dilatation",
  x2 = "Other-", x3 = "Surveillance",
  x4 = "APC", x5 = "Therapeutic- RFA TTS",
  x5 = "Therapeutic- RFA 90",
  x6 = "Therapeutic- EMR", x7 = "Therapeutic- RFA 360"
)
EndoEvent <- replicate(2000, sample(Event, 1, replace = FALSE))
# Merge the list with the Myendo dataframe
fff <- unlist(EndoEvent)
fff <- data.frame(fff)
names(fff) <- "col1"
Myendo$EndoEvent<-fff$col1
names(Myendo)[names(Myendo) == "HospitalNumber"] <- "PatientID"
names(Myendo)[names(Myendo) == "fff$col1"] <- "EndoEvent"
# Myendo$EndoEvent<-as.character(Myendo$EndoEvent)
# Run the function using the procedure information (the date of the
# procedure, the Event type and the individual patient IDs)
hh <- PatientFlow_CircosPlots(Myendo, "Dateofprocedure", "PatientID", "EndoEvent")
#> Using n as value column: use value.var to override.
#> Aggregation function missing: defaulting to length

rm(Myendo)
rm(EndoEvent)
```

--------------------------------

### DictionaryInPlaceReplace example with LocationList

Source: https://docs.ropensci.org/EndoMineR/reference/DictionaryInPlaceReplace.html

Example call to DictionaryInPlaceReplace using TheOGDReportFinal$OGDReportWhole and LocationList().

```r
inputText<-DictionaryInPlaceReplace(TheOGDReportFinal$OGDReportWhole,LocationList())
```

--------------------------------

### SurveilFirstTest Example

Source: https://docs.ropensci.org/EndoMineR/reference/SurveilFirstTest.html

Example usage of SurveilFirstTest applied to a dataframe named Myendo.

```R
dd <- SurveilFirstTest(
  Myendo, "HospitalNumber",
  "Dateofprocedure"
)
```

--------------------------------

### Example of EndoscInstrument usage

Source: https://docs.ropensci.org/EndoMineR/reference/EndoscInstrument.html

An example showing how to apply the EndoscInstrument function to a column in a dataframe.

```R
Myendo$Instrument <- EndoscInstrument(Myendo$Instrument)
```

--------------------------------

### HowManyOverTime example with Myendo

Source: https://docs.ropensci.org/EndoMineR/reference/HowManyOverTime.html

Example usage of HowManyOverTime. It clears the workspace, then calls the function on the Myendo dataframe, using the 'Indications' column for the indication, 'Dateofprocedure' for the date, and '.*' to match all indications. The result is stored in ff.

```r
# This takes the dataframe MyEndo (part of the package examples) and looks in
# the column which holds the test indication (in this example it is called
# 'Indication' The date of the procedure column(which can be date format or
# POSIX format) is also necessary.  Finally the string which indicates the text
# indication needs to be inpoutted. In this case we are looking for all endoscopies done
# where the indication is surveillance (so searching on 'Surv' will do fine).
# If you want all the tests then put '.*' instead of Surv
rm(list = ls(all = TRUE))
ff <- HowManyOverTime(Myendo, "Indications", "Dateofprocedure", ".*")
```

--------------------------------

### Extractor example with dividing words

Source: https://docs.ropensci.org/EndoMineR/reference/Extractor.html

Example usage of the Extractor function. A vector of dividing words is created (which will also act as column names) and passed along with the report column to Extractor. Note that column names cannot start with a number, so one of the dividing words has to be converted.

```r
# As column names cant start with a number, one of the dividing
# words has to be converted
# A list of dividing words (which will also act as column names)
# is then constructed
mywords<-c("Hospital Number","Patient Name:","DOB:","General Practitioner:",
"Date received:","Clinical Details:","Macroscopic description:",
"Histology:","Diagnosis:")
Mypath2<-Extractor(PathDataFrameFinal$PathReportWhole,mywords)
```

--------------------------------

### EndoscopyEvent example usage

Source: https://docs.ropensci.org/EndoMineR/reference/EndoscopyEvent.html

An example showing how to apply the EndoscopyEvent function to a dataframe column.

```R
# Myendo$EndoscopyEvent<-EndoscopyEvent(Myendo,"Findings",
# "ProcedurePerformed","MACROSCOPICALDESCRIPTION","HISTOLOGY")
```

--------------------------------

### EntityPairs_TwoSentence usage example

Source: https://docs.ropensci.org/EndoMineR/reference/EntityPairs_TwoSentence.html

An example showing how to call EntityPairs_TwoSentence with findings and event/histology lists.

```R
# tbb<-EntityPairs_TwoSentence(Myendo$Findings,EventList(),HistolType())
```

--------------------------------

### Endomerge2 usage example

Source: https://docs.ropensci.org/EndoMineR/reference/Endomerge2.html

Example showing how to merge the Myendo and Mypath dataframes using specific column names for dates and hospital numbers.

```R
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber",
  Mypath, "Dateofprocedure", "HospitalNumber"
)
```

--------------------------------

### Example usage of dev_ExtrapolateOPCS4Prep

Source: https://docs.ropensci.org/EndoMineR/reference/dev_ExtrapolateOPCS4Prep.html

Example usage of dev_ExtrapolateOPCS4Prep. Requires running HistolTypeSite and EndoscopyEvent functions first. The commented code assigns the result to SelfOGD_Dunn$OPCS4w.

```r
# Need to run the HistolTypeSite and EndoscopyEvent functions first here
# SelfOGD_Dunn$OPCS4w<-ExtrapolateOPCS4Prep(SelfOGD_Dunn,"PROCEDUREPERFORMED",
# "PathSite","EndoscopyEvent")
```

--------------------------------

### Example workflow for BarrettsBxQual

Source: https://docs.ropensci.org/EndoMineR/reference/BarrettsBxQual.html

Demonstrates the full workflow: cleaning histology data, merging with endoscopy data, running prerequisite Barrett's functions, and finally calling BarrettsBxQual. Includes warnings from the functions.

```r
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
Mypath$NumBx <- HistolNumbOfBx(Mypath$Macroscopicdescription, "specimen")
Mypath$BxSize <- HistolBxSize(Mypath$Macroscopicdescription)

# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", Mypath, "Dateofprocedure",
  "HospitalNumber"
)

# The function relies on the other Barrett's functions being run as well:
b1 <- Barretts_PragueScore(v, "Findings")
#> Warning: argument is not an atomic vector; coercing
b1$PathStage <- Barretts_PathStage(b1, "Histology")

# The follow-up group depends on the histology and the Prague score for a
# patient so it takes the processed Barrett's data and then looks in the
# Findings column for permutations of the Prague score.
b1$FU_Type <- Barretts_FUType(b1, "CStage", "MStage", "PathStage")
#> Warning: There were 6 warnings in `mutate()`.
#> The first warning was:
#> ℹ In argument: `FU_Type = case_when(...)`.
#> Caused by warning:
#> ! NAs introduced by coercion
#> ℹ Run `dplyr::last_dplyr_warnings()` to see the 5 remaining warnings.


colnames(b1)[colnames(b1) == "pHospitalNum"] <- "HospitalNumber"
# The number of average number of biopsies is then calculated and
# compared to the average Prague C score so that those who are taking
# too few biopsies can be determined
hh <- BarrettsBxQual(
  b1, "Date.x", "HospitalNumber",
  "Endoscopist"
)
#> Warning: NAs introduced by coercion
#> Warning: NAs introduced by coercion
rm(v)
```

--------------------------------

### BarrettsAll Example

Source: https://docs.ropensci.org/EndoMineR/reference/BarrettsAll.html

An example of calling BarrettsAll with endoscopy and pathology dataframes, including the expected warning output.

```R
Barretts_df <- BarrettsAll(Myendo, "Findings", "OGDReportWhole", Mypath, "Histology")
#> Warning: There were 6 warnings in `mutate()`.
#> The first warning was:
#> ℹ In argument: `FU_Type = case_when(...)`.
#> Caused by warning:
#> ! NAs introduced by coercion
#> ℹ Run `dplyr::last_dplyr_warnings()` to see the 5 remaining warnings.
```

--------------------------------

### CategoricalByEndoscopist example with data preparation

Source: https://docs.ropensci.org/EndoMineR/reference/CategoricalByEndoscopist.html

Example workflow for CategoricalByEndoscopist. It prepares histology data (number of biopsies, biopsy size), merges with endoscopy data using Endomerge2, computes IM status with Barretts_PathStage, then calls CategoricalByEndoscopist on the IM column and endoscopist column. The final rm(Myendo) produces a warning that the object is not found.

```r
# The function plots any numeric metric by endoscopist
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
v <- Mypath
v$NumBx <- HistolNumbOfBx(Mypath$Macroscopicdescription, "specimen")
v$BxSize <- HistolBxSize(v$Macroscopicdescription)
# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", v, "Dateofprocedure",
  "HospitalNumber"
)
# The function relies on the other Barrett's functions being run as well:
v$IMorNoIM <- Barretts_PathStage(v, "Histology")
colnames(v)[colnames(v) == "pHospitalNum"] <- "HospitalNumber"
# The function takes the column with the extracted worst grade of
# histopathology and returns the proportion of each finding (ie
# proportion with low grade dysplasia, high grade etc.) for each
# endoscopist
kk <- CategoricalByEndoscopist(v$IMorNoIM, v$Endoscopist)
rm(Myendo)
#> Warning: object 'Myendo' not found
```

--------------------------------

### Example: Barretts_PathStage on Mypath dataset

Source: https://docs.ropensci.org/EndoMineR/reference/Barretts_PathStage.html

Example usage of Barretts_PathStage on the Mypath demo dataset, extracting the worst histological grade from the Histology column. The example includes a call to rm(v) which produces a warning that object 'v' is not found.

```r
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
# The function then takes the Histology column from the merged data set (v).
# It extracts the worst histological grade for a specimen
b <- Barretts_PathStage(Mypath, "Histology")
rm(v)
#> Warning: object 'v' not found
```

--------------------------------

### HistolNumbOfBx example with Macroscopicdescription

Source: https://docs.ropensci.org/EndoMineR/reference/HistolNumbOfBx.html

Example call to HistolNumbOfBx that assigns the result to qq. It processes the Macroscopicdescription column of Mypath and uses "specimen" as the keyword to remove and stop at in the regex.

```r
qq <- HistolNumbOfBx(Mypath$Macroscopicdescription, "specimen")
```

--------------------------------

### SurveySankey Example

Source: https://docs.ropensci.org/EndoMineR/reference/SurveySankey.html

Example showing how to rename the hospital number column to PatientID and generate the Sankey plot.

```R
names(Myendo)[names(Myendo) == "HospitalNumber"] <- "PatientID"
gg <- SurveySankey(Myendo, "ProcedurePerformed", "PatientID")
#> starting httpd help server ...
#>  done
```

--------------------------------

### HistolTypeAndSite example

Source: https://docs.ropensci.org/EndoMineR/reference/HistolTypeAndSite.html

Demonstrates how to apply the function to a merged dataset containing pathology reports and procedure information.

```R
Myendo2<-Endomerge2(Myendo,'Dateofprocedure','HospitalNumber',
Mypath,'Dateofprocedure','HospitalNumber')
PathSiteAndType <- HistolTypeAndSite(Myendo2$PathReportWhole,
Myendo2$Macroscopicdescription, Myendo2$ProcedurePerformed)
```

--------------------------------

### NegativeRemoveWrapper example with sample text

Source: https://docs.ropensci.org/EndoMineR/reference/NegativeRemoveWrapper.html

Example that builds a character vector of sample medical text, converts it to a dataframe, and shows how to call NegativeRemoveWrapper on the column. The final call is commented out.

```r
# Build a character vector and then
# incorporate into a dataframe
anexample<-c("There is no evidence of polyp here",
"Although the prep was poor,there was no adenoma found",
"The colon was basically inflammed, but no polyp was seen",
"The Barrett's segment was not biopsied",
"The C0M7 stretch of Barrett's was flat")
anexample<-data.frame(anexample)
names(anexample)<-"Thecol"
# Run the function on the dataframe and it should get rid of sentences (and
# parts of sentences) with negative parts in them.
#hh<-NegativeRemoveWrapper(anexample$Thecol)
```

--------------------------------

### Extract histology from raw report using ExtrapolatefromDictionary

Source: https://docs.ropensci.org/EndoMineR/reference/ExtrapolatefromDictionary.html

Example that extracts histology from the raw report, standardises the histology terms through a series of regular expressions, and then extracts the type of tissue. The result is assigned to Mypath$Tissue. The example also includes a warning about an object not found.

```r
#Firstly we extract histology from the raw report
# The function then standardises the histology terms through a series of
# regular expressions and then extracts the type of tissue 
Mypath$Tissue<-suppressWarnings(
suppressMessages(
ExtrapolatefromDictionary(Mypath$Histology,HistolType()
)
)
)
rm(MypathExtraction)
#> Warning: object 'MypathExtraction' not found
```

--------------------------------

### MetricByEndoscopist example with EndoscMeds

Source: https://docs.ropensci.org/EndoMineR/reference/MetricByEndoscopist.html

Example from the documentation showing how to use MetricByEndoscopist. It first binds the output of EndoscMeds to the main dataframe, then calls MetricByEndoscopist with the 'Endoscopist' column and the 'Fent' metric. The commented line shows how to generate a graph with EndoBasicGraph. The final rm(Myendo) produces a warning that the object is not found.

```r
#The function gives a table with any numeric
# metric by endoscopist
# In this example we tabulate medication by
# endoscopist
# Lets bind the output of EndoscMeds to the main dataframe so we
# have a complete dataframe with all the meds extracted
MyendoNew<-cbind(EndoscMeds(Myendo$Medications),Myendo)

# Now lets look at the fentanly use per Endoscopist:
kk<-MetricByEndoscopist(MyendoNew,'Endoscopist','Fent')
#EndoBasicGraph(MyendoNew, "Endoscopist", "Fent") #run this
#if you want to see the graph
rm(Myendo)
#> Warning: object 'Myendo' not found
```

--------------------------------

### Calculating Time to Status with EndoMineR

Source: https://docs.ropensci.org/EndoMineR/reference/TimeToStatus.html

Example workflow demonstrating data preparation, merging histology and endoscopy datasets, and applying the TimeToStatus function.

```R
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
v <- Mypath
v$NumBx <- HistolNumbOfBx(v$Macroscopicdescription, "specimen")
v$BxSize <- HistolBxSize(v$Macroscopicdescription)

# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", v, "Dateofprocedure",
  "HospitalNumber"
)

# The function relies on the other Barrett's functions being run as well:
b1 <- Barretts_PragueScore(v, "Findings")
#> Warning: argument is not an atomic vector; coercing
b1$IMorNoIM <- Barretts_PathStage(b1, "Histology")
colnames(b1)[colnames(b1) == "pHospitalNum"] <- "HospitalNumber"

# The function groups the procedures by patient and gives
# all the procedures between
# the indicatorEvent amd the procedure just after the endpoint.
# Eg if the start is RFA and the
# endpoint is biopsies then it will give all RFA procedures and
# the first biopsy procedure

b1$EndoscopyEvent <- EndoscopyEvent(
  b1, "Findings", "ProcedurePerformed",
  "Macroscopicdescription", "Histology"
)
nn <- TimeToStatus(b1, "eHospitalNum", "EndoscopyEvent", "rfa", "dilat")
rm(v)
```

--------------------------------

### EndoscMeds example with cbind

Source: https://docs.ropensci.org/EndoMineR/reference/EndoscMeds.html

Example usage of EndoscMeds to clean the Medications column and combine with the original data frame.

```r
MyendoNew <- cbind(EndoscMeds(Myendo$Medications), Myendo)
```

--------------------------------

### textPrep example with column names

Source: https://docs.ropensci.org/EndoMineR/reference/textPrep.html

Example of using textPrep with a vector of column names and a pathology report column. The function cleans punctuation, checks spelling, maps terms, and lowercases the text.

```r
mywords<-c("Hospital Number","Patient Name:","DOB:","General Practitioner:",
"Date received:","Clinical Details:","Macroscopic description:",
"Histology:","Diagnosis:")
CleanResults<-textPrep(PathDataFrameFinal$PathReportWhole,mywords)
```

--------------------------------

### NegativeRemove example

Source: https://docs.ropensci.org/EndoMineR/reference/NegativeRemove.html

Example of building a character vector, converting it to a dataframe, and applying NegativeRemove to remove sentences with negative parts.

```r
# Build a character vector and then
# incorporate into a dataframe
anexample<-c("There is no evidence of polyp here",
"Although the prep was poor,there was no adenoma found",
"The colon was basically inflammed, but no polyp was seen",
"The Barrett's segment was not biopsied",
"The C0M7 stretch of Barrett's was flat")
anexample<-data.frame(anexample)
names(anexample)<-"Thecol"
# Run the function on the dataframe and it should get rid of sentences (and
# parts of sentences) with negative parts in them.
hh<-NegativeRemove(anexample$Thecol)
```

--------------------------------

### EndoPaste usage and example

Source: https://docs.ropensci.org/EndoMineR/reference/EndoPaste.html

Shows the function signature and a complete example with a sample dataframe and the resulting merged text output. The function takes a dataframe with pre-segregated columns and uses column headers as delimiters to recreate the full text. The output is a list containing a dataframe with one column of merged text and a character vector of delimiters for use with textPrep.

```R
EndoPaste(x)
```

```R
testList<-structure(list(PatientName = c("Tom Hardy", "Elma Fudd", "Bingo Man"
), HospitalNumber = c("H55435", "Y3425345", "Z343424"), Text = c("All bad. Not good", 
"Serious issues", "from a land far away")), class = "data.frame", row.names = c(NA, -3L))
EndoPaste(testList)
#> [[1]]
#>                                                                      X1_X2_X3
#> 1     PatientName Tom Hardy\nHospitalNumber H55435\nText All bad. Not good\n 
#> 2      PatientName Elma Fudd\nHospitalNumber Y3425345\nText Serious issues\n 
#> 3 PatientName Bingo Man\nHospitalNumber Z343424\nText from a land far away\n 
#> 
#> [[2]]
#> [1] "PatientName"    "HospitalNumber" "Text"          
#> 
```

--------------------------------

### Example: Extract Prague score from endoscopy dataset

Source: https://docs.ropensci.org/EndoMineR/reference/Barretts_PragueScore.html

Demonstrates applying Barretts_PragueScore to the Myendo dataset, searching the 'Findings' column for Prague score mentions. The example notes it is easiest to use on a Barrett's subset of data.

```r
# The example takes the endoscopy demo dataset and searches the
# Findings column (which contains endoscopy free text about the
# procedure itself). It then extracts the Prague score if relevant. I
# find it easiest to use this on a Barrett's subset of data rather than
# a dump of all endoscopies but of course this is a permissible dataset
# too


aa <- Barretts_PragueScore(Myendo, "Findings", "OGDReportWhole")
```

--------------------------------

### ListLookup example with myNotableWords and Myendo data frame

Source: https://docs.ropensci.org/EndoMineR/reference/ListLookup.html

Example usage of ListLookup. Defines a list of words of interest (myNotableWords) and calls ListLookup on the 'Findings' column of the Myendo data frame. The example can be applied to histopathology or endoscopic free text columns.

```r
# The function relies on defined a list of
# words you are interested in and then choosing the column you are
# interested in looking in for these words. This can be for histopathology
# free text columns or endoscopic. In this example it is for endoscopic
# columns
myNotableWords <- c("arrett", "oeliac")
jj <- ListLookup(Myendo, "Findings", myNotableWords)
```

--------------------------------

### SurveilTimeByRow Example

Source: https://docs.ropensci.org/EndoMineR/reference/SurveilTimeByRow.html

Example usage of SurveilTimeByRow applied to a dataframe named Myendo using specific column names for patient ID and procedure date.

```R
aa <- SurveilTimeByRow(
  Myendo, "HospitalNumber",
  "Dateofprocedure"
)
```

--------------------------------

### Example usage of GRS_Type_Assess_By_Unit

Source: https://docs.ropensci.org/EndoMineR/reference/GRS_Type_Assess_By_Unit.html

Example call to GRS_Type_Assess_By_Unit using the vColon dataframe with column names "ProcedurePerformed", "Endoscopist", "Diagnosis", and "Original.y".

```r
nn <- GRS_Type_Assess_By_Unit(
  vColon, "ProcedurePerformed",
  "Endoscopist", "Diagnosis", "Original.y"
)
```

--------------------------------

### EndoPaste example dataset and usage

Source: https://docs.ropensci.org/EndoMineR/articles/EndoMineR.html

Creates a sample dataframe and applies EndoPaste to merge columns into a single text column with delimiters. The result is a list where the first element is the data and the second is the delimiters.

```R
#An example dataset
testList<-structure(list(PatientName = c("Tom Hardy", "Elma Fudd", "Bingo Man"), 
                         HospitalNumber = c("H55435", "Y3425345", "Z343424"), Text = c("All bad. Not good", 
"Serious issues", "from a land far away")), class = "data.frame", row.names = c(NA, -3L))
myReadyDataset<-EndoPaste(testList)
```

--------------------------------

### SurveilLastTest example call

Source: https://docs.ropensci.org/EndoMineR/reference/SurveilLastTest.html

Example call to SurveilLastTest using a dataframe named Myendo, with the patient ID column "HospitalNumber" and the endoscopy date column "Dateofprocedure". The result is assigned to cc.

```r
cc <- SurveilLastTest(Myendo, "HospitalNumber", "Dateofprocedure")
```

--------------------------------

### BarrettsParisEMR usage example

Source: https://docs.ropensci.org/EndoMineR/reference/BarrettsParisEMR.html

Example usage of BarrettsParisEMR. It assigns the result to a new column EMR in the Myendo data frame, using the ProcedurePerformed and Findings columns as inputs.

```r
# 
Myendo$EMR<-BarrettsParisEMR(Myendo$ProcedurePerformed,Myendo$Findings)
```

--------------------------------

### EndoBasicGraph example

Source: https://docs.ropensci.org/EndoMineR/reference/EndoBasicGraph.html

Example usage of EndoBasicGraph plotting numeric y vs non-numeric x. It creates numeric columns from histology data and then calls the function. The output includes warnings about legend.margin and geom_smooth formula.

```r
# This function plots numeric y vs non-numeric x
# Get some numeric columns e.g. number of biopsies and size
Mypath$Size <- HistolBxSize(Mypath$Macroscopicdescription)
Mypath$NumBx <- HistolNumbOfBx(Mypath$Macroscopicdescription, "specimen")
Mypath2 <- Mypath[, c("NumBx", "Size")]
EndoBasicGraph(Mypath, "Size", "NumBx")
#> Warning: `legend.margin` must be specified using `margin()`
#> ℹ For the old behavior use `legend.spacing`
#> `geom_smooth()` using formula = 'y ~ x'
```

--------------------------------

### Example of cleaning the endoscopist column

Source: https://docs.ropensci.org/EndoMineR/reference/EndoscEndoscopist.html

Demonstrates applying the EndoscEndoscopist function to a column within a dataframe.

```R
Myendo$Endoscopist <- EndoscEndoscopist(Myendo$Endoscopist)
```

--------------------------------

### Create Sankey plot of patient flow with SurveySankey

Source: https://docs.ropensci.org/EndoMineR/articles/Analysis.html

Commented-out call to SurveySankey that would create a Sankey plot of patient flow based on the 'ProcedurePerformed' column. The example is commented out, so it is not executed.

```r
#how<-SurveySankey(Myendo,"ProcedurePerformed")
```

--------------------------------

### EventList function usage

Source: https://docs.ropensci.org/EndoMineR/reference/EventList.html

Call EventList() with no arguments to get the standardised event list. This function is part of the NLP Lexicons group.

```r
EventList()
```

--------------------------------

### Check documentation quality with ListLookup

Source: https://docs.ropensci.org/EndoMineR/articles/Analysis.html

Loads the tm package, defines a vector of notable words (barrett, coeliac), and calls ListLookup on the 'Findings' column to get the proportion of reports containing each word. The output shows that barrett appears in 18.05% and coeliac in 6.20% of reports.

```r
library(tm)
myNotableWords <- c("barrett", "coeliac")
ListLookup(Myendo,'Findings',myNotableWords)
```

--------------------------------

### PatientFlowIndividual example for Barrett's progression

Source: https://docs.ropensci.org/EndoMineR/reference/PatientFlowIndividual.html

Example usage of PatientFlowIndividual for visualizing Barrett's progression over time. The commented code shows how to prepare the data with BarrettsAll and call the function with notable words for categorical outcomes. After running the function, call dev.off() to close the graphics device.

```r
# This function builds chart of categorical outcomes for individal patients over time
# It allows a two dimensional visualisation of patient progress. A perfect example is 
# visualising the Barrett's progression for patients on surveillance and then
# therapy if dysplasia develops and highlighting recurrence if it happens
# Barretts_df <- BarrettsAll(Myendo, "Findings", "OGDReportWhole", Mypath, "Histology")
# myNotableWords<-c("No_IM","IM","LGD","HGD","T1a","IGD","SM1","SM2")
# PatientFlowIndividual(Barretts_df,"IMorNoIM",myNotableWords,DateofProcedure,"HospitalNumber")
# Once the function is run you should always call dev.off()
```

--------------------------------

### TimeToStatus(dataframe, HospNum, EVENT, indicatorEvent, endEvent)

Source: https://docs.ropensci.org/EndoMineR/reference/TimeToStatus.html

Selects patients who have had a start event and an end event, allowing users to determine the time between two events, such as time to a fully squamous oesophagus after Barrett's ablation.

```APIDOC
## TimeToStatus(dataframe, HospNum, EVENT, indicatorEvent, endEvent)

### Description
This function selects patients who have had a start event and an end event of the users choosing so you can determine things like how long it takes to get a certain outcome. For example, how long does it take to get a patient into a fully squamous oesophagus after Barrett's ablation for dysplasia?

### Usage
```r
TimeToStatus(dataframe, HospNum, EVENT, indicatorEvent, endEvent)
```

### Arguments
- **dataframe** - The dataframe.
- **HospNum** - The Hospital Number column.
- **EVENT** - The column that contains the outcome of choice.
- **indicatorEvent** - The name of the start event (can be a regular expression).
- **endEvent** - The name of the endpoint (can be a regular expression).

### See also
Other Basic Analysis - Surveillance Functions: `HowManyOverTime()`, `SurveilFirstTest()`, `SurveilLastTest()`, `SurveilTimeByRow()`

### Examples
```r
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
v <- Mypath
v$NumBx <- HistolNumbOfBx(v$Macroscopicdescription, "specimen")
v$BxSize <- HistolBxSize(v$Macroscopicdescription)

# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", v, "Dateofprocedure",
  "HospitalNumber"
)

# The function relies on the other Barrett's functions being run as well:
b1 <- Barretts_PragueScore(v, "Findings")
#> Warning: argument is not an atomic vector; coercing
b1$IMorNoIM <- Barretts_PathStage(b1, "Histology")
colnames(b1)[colnames(b1) == "pHospitalNum"] <- "HospitalNumber"

# The function groups the procedures by patient and gives
# all the procedures between
# the indicatorEvent and the procedure just after the endpoint.
# Eg if the start is RFA and the
# endpoint is biopsies then it will give all RFA procedures and
# the first biopsy procedure

b1$EndoscopyEvent <- EndoscopyEvent(
  b1, "Findings", "ProcedurePerformed",
  "Macroscopicdescription", "Histology"
)
nn <- TimeToStatus(b1, "eHospitalNum", "EndoscopyEvent", "rfa", "dilat")
rm(v)
```

```

--------------------------------

### MyImgLibrary Usage

Source: https://docs.ropensci.org/EndoMineR/reference/MyImgLibrary.html

Basic syntax for calling the MyImgLibrary function.

```R
MyImgLibrary(file, delim, location)
```

--------------------------------

### textPrep(inputText, delim)

Source: https://docs.ropensci.org/EndoMineR/reference/textPrep.html

This function prepares the data by cleaning punctuation, checking spelling against the lexicons, mapping terms according to the lexicons and lower casing everything. It contains several of the other functions in the package for ease of use.

```APIDOC
## textPrep(inputText, delim)

### Description
This function prepares the data by cleaning punctuation, checking spelling against the lexicons, mapping terms according to the lexicons and lower casing everything. It contains several of the other functions in the package for ease of use.

### Usage
```
textPrep(inputText, delim)
```

### Arguments
- **inputText** - The relevant pathology text columns
- **delim** - the delimitors so the extractor can be used

### Value
This returns a string vector.

### Examples
```
mywords<-c("Hospital Number","Patient Name:","DOB:","General Practitioner:",
"Date received:","Clinical Details:","Macroscopic description:",
"Histology:","Diagnosis:")
CleanResults<-textPrep(PathDataFrameFinal$PathReportWhole,mywords)
```
```

--------------------------------

### ExtrapolatefromDictionary usage

Source: https://docs.ropensci.org/EndoMineR/reference/ExtrapolatefromDictionary.html

Shows the function signature with two arguments: inputString (the text string to process) and list (of words to iterate through).

```r
ExtrapolatefromDictionary(inputString, list)
```

--------------------------------

### Prepare pathology data for biopsy extraction

Source: https://docs.ropensci.org/EndoMineR/articles/EndoMineR.html

Creates a dataframe containing hospital number, patient name, and macroscopic description for further processing.

```R
sg<-data.frame(Mypath$HospitalNumber,Mypath$PatientName,Mypath$Macroscopicdescription)
```

--------------------------------

### HistolBxSize function usage

Source: https://docs.ropensci.org/EndoMineR/reference/HistolBxSize.html

Shows the function signature and arguments. The MacroColumn argument is typically the Macroscopic description column.

```r
HistolBxSize(MacroColumn)
```

--------------------------------

### GISymptomsList()

Source: https://docs.ropensci.org/EndoMineR/reference/GISymptomsList.html

Returns all common GI symptoms as a list. The symptoms are listed without grouping or mapping, derived from a manual list with synonyms from the UMLS Metathesaurus.

```APIDOC
## GISymptomsList()

### Description
Returns all common GI symptoms as a list. The symptoms are listed without grouping or mapping, derived from a manual list with synonyms from the UMLS Metathesaurus.

### Usage
```r
GISymptomsList()
```

### Arguments
None.

### Value
A character vector of common GI symptoms.

### See also
Other NLP - Lexicons: `BiopsyIndex()`, `EventList()`, `HistolType()`, `LocationListLower()`, `LocationListUniversal()`, `LocationListUpper()`, `LocationList()`, `RFACath()`, `WordsToNumbers()`
```

--------------------------------

### TimeToStatus Usage Syntax

Source: https://docs.ropensci.org/EndoMineR/reference/TimeToStatus.html

The basic function signature for calculating time between events.

```R
TimeToStatus(dataframe, HospNum, EVENT, indicatorEvent, endEvent)
```

--------------------------------

### DictionaryInPlaceReplace function usage

Source: https://docs.ropensci.org/EndoMineR/reference/DictionaryInPlaceReplace.html

Shows the function signature with arguments inputString and list.

```r
DictionaryInPlaceReplace(inputString, list)
```

--------------------------------

### ColumnCleanUp example: clean Myendo$Findings

Source: https://docs.ropensci.org/EndoMineR/reference/ColumnCleanUp.html

Applies ColumnCleanUp to the Findings column of the Myendo data frame. The function returns a cleaned character vector.

```R
ii<-ColumnCleanUp(Myendo$Findings)
```

--------------------------------

### SurveySankey Usage

Source: https://docs.ropensci.org/EndoMineR/reference/SurveySankey.html

Basic syntax for calling the SurveySankey function.

```R
SurveySankey(dfw, ProcPerformedColumn, PatientID)
```

--------------------------------

### Prepare sample pathology report for extraction

Source: https://docs.ropensci.org/EndoMineR/articles/EndoMineR.html

Renames the sample dataset column to 'PathReportWhole' and displays the first row with pander. This is the input for the Extractor function.

```R
PathDataFrameFinalColon2<-PathDataFrameFinalColon
names(PathDataFrameFinalColon2)<-"PathReportWhole"
pander(head(PathDataFrameFinalColon2,1))
```

--------------------------------

### MyImgLibrary(file, delim, location)

Source: https://docs.ropensci.org/EndoMineR/reference/MyImgLibrary.html

This function extracts and cleans endoscopic images from HTML exports, preparing them for linking to pathology and endoscopy reports. It takes an HTML report, a delimiter phrase, and a folder location as inputs.

```APIDOC
## MyImgLibrary(file, delim, location)

### Description
This function is used to pick and clean endoscopic images from html exports so they can be prepared before being linked to pathology and endoscopy reports.

### Usage
```r
MyImgLibrary(file, delim, location)
```

### Arguments
- **file** - The html report to extract (the html will have all the images references in it)
- **delim** - The phrase that separates individual endoscopies
- **location** - The folder containing the actual images

### Examples
```r
# MyImgLibrary("~/Images Captured with Proc Data Audit_Findings1.html",
#                         "procedureperformed","~/")
```
```

--------------------------------

### HowManyOverTime function signature

Source: https://docs.ropensci.org/EndoMineR/reference/HowManyOverTime.html

Function signature for HowManyOverTime. It takes a dataframe, the column name for the indication, the column containing the procedure date, and a string to search for in the indication column. Returns a list with a plot and a table of counts by month and year.

```r
HowManyOverTime(dataframe, Indication, Endo_ResultPerformed, StringToSearch)
```

--------------------------------

### SurveilFirstTest Usage

Source: https://docs.ropensci.org/EndoMineR/reference/SurveilFirstTest.html

Basic syntax for calling the SurveilFirstTest function with a dataframe, patient ID column, and procedure date column.

```R
SurveilFirstTest(dataframe, HospNum_Id, Endo_ResultPerformed)
```

--------------------------------

### Define delimiters and extract fields with EndoMineR::Extractor

Source: https://docs.ropensci.org/EndoMineR/articles/EndoMineR.html

Loads EndoMineR, defines the delimiters (column headers) to split the report, and calls Extractor to return a dataframe with columns containing the text between delimiters.

```R
library(EndoMineR)
mywords<-c("Hospital Number","Patient Name:","DOB:","General Practitioner:",
"Date received:","Clinical Details:","Macroscopic description:",
"Histology:","Diagnosis:")

PathDataFrameFinalColon3<-Extractor(PathDataFrameFinalColon2$PathReportWhole,mywords)
```

--------------------------------

### Usage of sanity function

Source: https://docs.ropensci.org/EndoMineR/reference/sanity.html

The basic syntax for calling the sanity function.

```R
sanity(pathName)
```

--------------------------------

### Usage of GISymptomsList

Source: https://docs.ropensci.org/EndoMineR/reference/GISymptomsList.html

Call this function to retrieve the complete list of common GI symptoms.

```R
GISymptomsList()
```

--------------------------------

### Eosinophilics(dataframe, findings, histol, IndicationsFroExamination)

Source: https://docs.ropensci.org/EndoMineR/reference/Eosinophilics.html

Extracts the Prague score (C and M stages) for Barrett's samples from free text using a regex where C and M stages are explicitly mentioned.

```APIDOC
## Eosinophilics(dataframe, findings, histol, IndicationsFroExamination)

### Description
Extracts the Prague score (C and M stages) for Barrett's samples from free text using a regex where C and M stages are explicitly mentioned.

### Usage
```r
Eosinophilics(dataframe, findings, histol, IndicationsFroExamination)
```

### Arguments
- **dataframe** (dataframe) - Required - Dataframe with column of interest.
- **findings** (column) - Required - Column of interest.
- **histol** (column) - Required - Second column of interest.
- **IndicationsFroExamination** (column) - Required - Second column of interest.

### Examples
```r
# Firstly relevant columns are extrapolated from the
# Mypath demo dataset. These functions are all part of Histology data
# cleaning as part of the package.
v <- Mypath
v$NumBx <- HistolNumbOfBx(v$Macroscopicdescription, "specimen")
v$BxSize <- HistolBxSize(v$Macroscopicdescription)
# The histology is then merged with the Endoscopy dataset. The merge occurs
# according to date and Hospital number
v <- Endomerge2(
  Myendo, "Dateofprocedure", "HospitalNumber", v, "Dateofprocedure",
  "HospitalNumber"
)

aa <- Eosinophilics(v, "Findings", "Histology", "Indications")
```
```

--------------------------------

### Count indications over time with HowManyOverTime

Source: https://docs.ropensci.org/EndoMineR/articles/Analysis.html

Counts how many times each indication occurs over time, grouped by the date of procedure and the endoscopist (Surv). The result is stored in the variable 'how'.

```r
how<-HowManyOverTime(Myendo,'Indications','Dateofprocedure','Surv')
```

--------------------------------

### ColumnCleanUp usage

Source: https://docs.ropensci.org/EndoMineR/reference/ColumnCleanUp.html

Shows the required arguments for the ColumnCleanUp function. The only argument is 'vector', the column of interest.

```R
ColumnCleanUp(vector)
```

--------------------------------

### DictionaryInPlaceReplace(inputString, list)

Source: https://docs.ropensci.org/EndoMineR/reference/DictionaryInPlaceReplace.html

Maps terms in the input text and replaces them with the standardized term from the provided list. Used within the textPrep function for text cleaning.

```APIDOC
## DictionaryInPlaceReplace(inputString, list)

### Description
Maps terms in the text and replaces them with the standardized term (mapped in the lexicon file) within the text. Used within the textPrep function.

### Usage
```r
DictionaryInPlaceReplace(inputString, list)
```

### Arguments
- **inputString** (character) - Required - The input string (i.e., the full medical report).
- **list** (list) - Required - The replacing list.

### Value
Returns a character vector.

### Examples
```r
inputText <- DictionaryInPlaceReplace(TheOGDReportFinal$OGDReportWhole, LocationList())
```

### See Also
Other NLP - Text Cleaning and Extraction: `ColumnCleanUp()`, `Extractor()`, `NegativeRemoveWrapper()`, `NegativeRemove()`, `textPrep()`
```

--------------------------------

### Usage of spellCheck function

Source: https://docs.ropensci.org/EndoMineR/reference/spellCheck.html

The basic syntax for the spellCheck function, which takes a pattern, a replacement, and the target string.

```R
spellCheck(pattern, replacement, x, fixed = FALSE)
```

--------------------------------

### Usage of EndoscInstrument

Source: https://docs.ropensci.org/EndoMineR/reference/EndoscInstrument.html

The basic syntax for calling the EndoscInstrument function.

```R
EndoscInstrument(EndoInstrument)
```