\documentclass{StyleFile}

\usepackage[margin=1in]{geometry}
% \usepackage{ulem}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{lastpage, bbding, pmboxdraw}
\usepackage{amsfonts}
%\usepackage[sorting=none]{biblatex}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}

\input{macros.tex}
\addbibresource{refs.bib}



\begin{document}
\newcommand{\papertitle}{
Inferring Evidence-Based Processes from Outcomes and History}

%\newcommand{\papertitle}{Clinical Decision Support System for diagnosis and treatment of patients in ICU}
\newcommand{\name}{Shantanu Bhatia, Yana Chala}
\newcommand{\advisor}{Azza Abouzied, Dennis Shasha}

\title{\papertitle}

\begin{center}
\bf \large \name, Advisors: \advisor \\
\vskip10pt
%\papertitle
\vskip10pt
\hrule
\end{center}





\section*{Abstract}
In this paper, we propose a system for analyzing Intensive Care Unit (ICU) visits and thereby supporting the diagnosis and treatment of patients in ICU settings. The goal of the system is to infer recommendations for actions that medical staff should perform on a given patient. To achieve this, we find patients in similar conditions, and devise a number of strategies for recommending actions that should improve a patient's medical state. These recommendations are derived from the set of similar patients and a patient's most recent test results. Currently, our system is trained on data collected from ICU patients admitted for pneumonia. In the future, it can be extended to accommodate a  variety of diseases. We compared our system against a random sampling of actions. Our highest-performing strategy yields a mean F-score of $0.417$ after three days, outperforming the mean F-score of the random sampler by $0.150$ (p-value $0.026$).  
% \dennis{You can say something about the results. How you improve compared to the baseline and whether statistically significant.}


\section{Introduction and Goals}
\subsection{Background}
The demand for evidence-based medical systems that can help doctors determine diagnoses and treatments is growing and becoming widely adapted in the medical fields. Hence, there is a need for clinical decision support systems that could help find effective treatment for Intensive Care Unit (ICU) patients in time-sensitive scenarios. We assume that an ICU patient's visit consists of measurements that give medical information, and actions that are taken to treat the patient. From this, we assume that actions are taken as a result of what values are returned by these measurements. Therefore, our goal is to recommend actions a patient requires in order to be discharged from the ICU, based on their vital sign information and test results. 

\subsection{Data Sources}
For this study, we used the Medical Information Mart for
Intensive Care (MIMIC-III) database, a large, open-access dataset of patients admitted to the Beth Israel Deaconess Medical
Center between 2001 and 2012. This database contains over 58,000 hospital admissions for 38,645 adults and 7,875 neonates and is hosted by the Laboratory for Computational Physiology at the Massachusetts Institute of Technology.
MIMIC-III contains data that were downloaded from different sources, including archives from critical care information systems, hospital electronic health record databases, and out-of-hospital records of patient mortality. Such data include high resolution vital signs, laboratory results, prescribed medications, fluid balance, procedure codes, diagnostic codes, imaging reports, provider notes, and hospital administrative data. All these data were collected during routine clinical care and data collection was not visible to caregivers, to prevent interference with their work flow. To focus our study, we focus only on records generated by hospital admissions diagnosed with pneumonia which contains 1566 patients. Out of these we looked at data obtained from the MetaVision dataset because it includes start and end time for actions, giving us a sample of 969 patients. 
\cite{mimic}. 


\subsection{Motivating Example - Pneumonia}
%Each year approximately 900,000 children aged $<5$ years die from pneumonia worldwide; $70\%$ of childhood pneumonia hospitalizations in the United States are in this age group \cite{stats1}. 
%After exploring and comparing available data we decided to focus on ICU admissions for pneumonia. 
Our dataset of 1566 hospital admissions for pneuomonia contains 963 patients who were cured and discharged, and 609 patients who died. Other diseases contained either fewer total entries or had more unbalanced mortality rates.
Pneumonia is a leading infectious cause of hospitalization and death among U.S. adults, resulting in more than \textdollar10 billion in hospital expenses in 2011 \cite{stats2}. Significant life savings can be achieved if accurate treatments can be determined within shorter time frames. 
 %Our reasoning for choosing a disease with a relatively even mortality rate was to choose such disease that is treatable, however, in many cases patients die due to various preventable reasons. 


\subsection{Basic Definitions}

We define some keywords related to our data model as follows:

{\bf Definition: } A \textbf{test} is a measurement performed to detect, diagnose, or monitor disease state.  For example, temperature measure, white blood cell count, blood oxygen level.

{\bf Definition: } An \textbf{action} refers to medical treatment or procedure that can be performed on a patient in an attempt to improve a patient's health state. For example, a course of antibiotics or eye surgery.

{\bf Definition: } A \textbf{state} is a set of a patient's most recent test results at a particular time.

%{\bf Definition: } A state $s1$ is \textbf{sub-consistent} with a state $s2$ if the test results of $s1$ are a subset of those of $s2$ or are approximately (in a sense to be defined later) a subset. We also say that $s2$ is \textbf{super-consistent} with $s1$.

{\bf Definition: } A \textbf{good state} is a set of test results that indicate that the patient is healthy enough to be discharged. 

{\bf Definition:} A \textbf{bad state} is a set of test results that indicate that the patient will die with high probability.

{\bf Definition:} A \textbf{triple} is a tuple indicating the observed result of an action on a particular measurement's value. It indicates the last recorded value for the measurement before the action and the first recorded value for the measurement after the action.

{\bf Definition:} An \textbf{abnormal} is a test result whose value lies outside of the 90\% confidence interval around the median  value for that test among patients just before discharge (informally, healthy patients).

\subsection{Methods at a High Level}

We first attempt to establish a causal relationship between measurements and actions. Actions map states to other states. Ideally, they would help transfer a patient from a bad state to a good state. In order to determine such a mapping, we construct triples, i.e. look at all the patients that have already had actions done to them and record their states right before and right after the action took place.

Each test has some `healthy' range that its results must lie in. By looking at the last recorded test results of patients who were  discharged, we can determine the median value for each test result, as well as a 90\% confidence interval around the median. Any value outside this confidence interval is considered to be an abnormal value.

To infer the actions to take given a state, we look at the patient's state and find all the abnormal values. We use these abnormal values to find similar states among other pneumonia patients. The actions corresponding to these states give us a list of actions that were done to patients in a similar condition. We then employ a variety of strategies for picking a subset of these actions as our set of recommendations. This methodology is illustrated in Figure \ref{fig:highLevelVis}.

While we do not currently determine which tests are required, a possible way to do so would be to use our machine learning models that predict if an action is required to also identify important features (tests) that might determine whether or not to perform an action (e.g. bacterial count tests before applying an antibiotic). 


\begin{figure}[H]
\centering
\includegraphics[height=8.5cm]{figures/KNN_Abnormal_Visualisation.png}
\caption{Visualization of high-level methodology. On the left, the patient being monitored has some recent measurements comprising their current state, of which we identify two abnormal values (highlighted in red). From these abnormals, we can find patients with similar values for the abnormals, as shown on the right. Note that the other values in similar patients' states are not considered when identifying nearest neighbours. From the actions taken on these similar patients when they were in these states, we can now apply one of the strategies presented in Section \ref{identifyingActions} to choose actions needed for the monitored patient.}
\label{fig:highLevelVis}
\end{figure}


%If our patient lacks any of these important features, those tests are required to be taken.





\section{Related Work}
%More thematic in future rather then specifically talking about the particular work. For now, move to the end.
% Doctors in hospitals may not encounter specific diseases very often, and different doctors would have varying levels of experience with different diseases. A clinical decision support system like ours aims to help doctors to see an entire overview of a patient's health as well as identifying key features that might be overlooked. 
We mainly studied existing approaches to modeling and predicting results of patient data. We  attempted to find systems that either try to learn trends from clinical time-series data or served as clinical decision support systems for recommending actions and tests in time-sensitive scenarios. We also attempted to find systems that used similar methods to ours in order to assess their effectiveness, namely, modeling clinical time stamped data, establishing similarity between the patients, and predicting actions and tests that could improve patient's health.

\subsection{ Data abstraction }
Clinical time stamped databases often contain a lot of data that is inherently interconnected and complex. As a consequence anyone who attempts to perform intelligent data analysis of clinical time-stamped data inevitably faces the challenge of representing such data in the most useful and efficient way. There exist a number of techniques to represent such data including temporal abstractions, time-series representations and machine-generated natural-language summaries. 

\cite{Multivariate} and \cite{TemporalPatterns} model patient data as time series. \cite{TemporalPatterns} aims to diagnose patients from the time series of ECG data by identifying abnormal trends in the series. They also aim to provide interpretable estimates of temporal uncertainty on the diagnosis. However, the time series used are of a single measurement. \cite{Multivariate} presents a method of representing patient data as a multivariate time series across several different measurements, and finding important features in the time series. They use this multivariate time series to make early diagnoses. While these papers try to accomplish a different goal to us, they too identify abnormal values among test results, which lead them to a classification. We similarly learn from patients abnormal values but we go a step further by recommending actions to resolve these abnormals.


Shahar \cite{ShaharTA} defines temporal abstraction (TA) as ``a process which, given a set of time-stamped parameters, external events and abstraction goals, produces abstractions of the data that interpret past and present states and trends and that are relevant for the given set of goals" \cite{TemporalAbstraction}. Temporal abstraction goals may vary from representing simple level shifts and trends to representing more complex phenomena such as rapid heart rate decelerations that could be of vast assistance to clinical data analysis. Stacey and McGregor \cite{TemporalAbstraction} analyzed 14 systems that utilize various mechanisms of temporal abstraction in clinical setting. They emphasize the importance of integrating the knowledge acuired from such abstraction with other data mining techniques performed when analyzing clinical data. While we currently do not perform temporal abstraction in our system, we believe that it could benifit from integration of TA rules and mechanisms in future.

An alternative approach to data abstraction is to provide natural-language summaries for medical professionals to read and base their course of action on. Hunter et al \cite{NatLang} studied the effectiveness of using textual summaries of data generated in neonatal care wards. Nurses and doctors were shown to be more effective at absorbing a days' worth of measurement and action data from a specific ward through an English-language summary than from the raw data or data visualization. However, human-written summaries still outperformed summaries written using NLP techniques. If we attempt to generate textual summaries, we would be able to provide more comprehensive reports, as we would be able to summarize which measurements have improved or gotten worse on a patient over a time period. We would also be able to directly constrast the actions already taken against actions we recommend. Our system could potentially also be used to generate informative reports on patient state and abnormals, which would be useful to doctors.


\subsection{ Similarity metric between the patients }
In an attempt to develop a system that could recomend actions and states solely from data without intervention of domain knowledge experts, it is useful to be able to recognize similar patients. Analyzing the ways in which similar patients get treated could help treat future patients. Thus, in this section we look at different similarity metrics among the patients.

%Since we're dealing with time series medical data, we looked at paper that tried to do simialrity in the same context
%Reference the papers that do dat, descire em w/ their method

There exist several drawbacks to existing similarity calculation methods. Firstly, similarity metrics for medical data can be arbitrarily defined and not directly relevant - for example, demographic data. Secondly, when patient data is modeled as a time series, finding similar series' can be difficult if the patient's time series is especially short. That is to say, given a new patient, existing time series similarity metrics are not effective at identifying ``neighbour" patients. We solve this issue by finding similar patients at a specific state, which is at a particular point in time and not dependent on the rest of the series \cite{Multivariate}.

An example of similarity metric that finds similar patients at a particular point in time is introduced in \cite{Ebadollahi2010}. Here, the authors tried to adjust feature importance by leveraging domain knowledge recorded as labels from doctors. Quantitive measurements of a patient are represented as N-dimensional vectors. Then they they use Mahalanobis distance to find similarity between ${x_i}$ and ${x_j}$ defined as 
\begin{equation}
d_{m}(x_i, x_j) = \sqrt{(x_i - x_j)^{T}P(x_i-x_j)}
\end{equation}

by finding optimal $P\in\mathbb{R}^{N\times N}$, such that the patients that have the same labels are closer together while patients with different labels are far away. Their similarity metric is then evaluated based on ability to predict domain knowledge label from the N-dimensional vectors. 

While Ebadollahi et al. achieve high accuracy and precision when predicting whether a patient will experience an Acute Hypotensive Episode, which served as a label in their experiment, such a similarity metric would not suffice if there are no clear labels available. In our scenario, we are interested in finding patients that are similar to a given patient purely based on the patient's test measurements, without placing them in buckets defined by labels. That is why we devise our own similarity measure described in the methodology section below.

\subsection{ Recommending Tests and Actions }
There currently exist several Clinical Decision Support Systems (CDSS) for diagnosing patients with a specific disease \cite{AptaSDSS, Dementia, DiagTree}. These systems are also capable of identifying key tests that would aid in diagnosis Typically, these systems rely on human-defined rules for their diagnosis. In \cite{Dementia}, a system is proposed for using senior doctors' diagnosis of potential dementia patients to identify key features in dementia diagnosis. This allows less experienced doctors to note and record the measurements for these key features. \cite{AptaSDSS} proposes a CDSS that uses an ensemble of machine learning methods for classifying and diagnosing cardiovascular disease in patients. The training data here consists of labeled aptamer chips made from blood samples of patients. The classifiers were trained on a dataset with a feature dimensionality of 3000 features. From these, the 250 features with the highest significance score (p-value) were chosen, and AptaCDSS-E uses the values for these tests to diagnose patients. However, the system is unable to adapt to having insufficient input data, and is unable to either make a diagnosis or recommend further tests required. The authors note that when the system, designed for diagnosing CVDs, was tested on data of patients with tuberculosis, accuracy of predictions was low due to poor quality of data. 

Finally, \cite{DiagTree} discusses a system to identify useful tests for diagnosing a given disease. Given a knowledge state and a possible set of tests that can be conducted, DiagTree determines which tests are most likely to reduce the uncertainty of a diagnosis. It uses greedy heuristics and an integer-programming based solution to determine the most useful test at every step in the tree. By contrast, our approach is more opportunistic: it identifies those test results that determine the actions required. If two diagnoses would lead to the same actions, then we do not distinguish among the diagnoses. Further, our system aims to be generalizable to the treatment of any disease.

\section{Methodology}
%After familiarizing ourselves with the schema and contents of MIMIC-III and deciding to pursue pneumonia treatment, we first
Our system encodes test and action data in a form more useful to us than present by default in MIIMIC-III. The processing pipeline is split into the following steps:
\begin{enumerate}
    \setcounter{enumi}{-1} %This is so that the list starts at 0 instead of 1%
    \item 
  Characterize good and bad states based on the test results that result in discharge (good) or death (bad).
    \item
    Compile action-state data to see how an action affects patient state.
    \item
    Determine median healthy values for tests (from the values in good states).
    \item
    Find abnormal values among the patient's current test results.
    \item
    Find  states (from other patients) that are similar to our patient's abnormal test results.
    \item
    From these similar states, find potential actions.
    \item
    Based on one of the strategies detailed in Section \ref{identifyingActions}, recommend a set of actions that need to be done to the patient.
\end{enumerate}

%In addition, we trained a decision tree classifier to understand which test are important and can help predict if the patient is likely to survive.
%Lastly, we designed future steps needed to complete out decision support system, including finding patients with similar test results that were later treated, identifying the best action to take. These steps are discussed further in the next section.

\subsection{Data encoding}
\label{Data encoding}
Some of the tests do not have numerical value results, but rather represent results as strings. We encode some of them using one-hot encoding if these are categorical variables. Otherwise, we use ordinal encoding. 
\begin{enumerate}
    \item
    \label{Encoding}
    \textbf{Ordinal Encoding}. Some of the results have a natural order and are encoded as strings, for instance, Skin Temperature -`Hot', `Warm', `Cool'. We encode test results of similar type  using ordinal encoding. Meaning we do not add any dimensions, but rather we assign an integer value to each category.
    \item
    \label{one-hot}
    \textbf{One-Hot encoding}. However, some tests do not have a natural order to them. For example, the test Pain Type can take the values `Dull', `Burning', `Intermittent', `Sharp', `Constant', `Stabbing', `Gassy', `Aching', `Shooting', `Throbbing', `Cramping', `Nagging'. This is an example of a categorical variable. We encode categorical variables using One-Hot encoding. Before the encoding we replace missing values with the most frequent value. %{\em Dennis thinks we should store these as nulls as we might need to do this test. We can assume the most frequent value when building the decision tree for good and bad states.} 
    As the result of this encoding we will have as many additional dimensions as there are categories.
    \item
    \label{one-hot}
    \textbf{Missing values.} During testing, we replace missing test values with the means for those tests, except for the binary encoded variables, as these already have the missing values replaced with the most frequent values. 
\end{enumerate}

Apart from the encoding of tests with non-numerical and non-ordinal values, there was also the issue of the same test or action having multiple entries in MIMIC-III's dictionary of items. For example, the test for White Blood Cell count has five different entries, and the same patient may have two different measurements of their WBC count listed as different tests, as a result. Therefore, we determine a new set of keys for items, where all the separate entries for the same action or test are assigned the same common ID. This is achieved by normalizing item labels. We first make all labels lowercase and remove any whitespace. We then eliminate any punctuation, trailing digits, parentheses and percentage signs. For example, the fluid-input actions `Saline 0.9\%' and `Saline 0.45\%' are both normalized to `saline'. We then assign new item IDs, giving the same itemID to all items with the same normalized label. By this method, out of 12487 items (including both actions and tests), 3104 items were found to be repeats of already-existing items, leaving us with 9383 distinct items. 

\subsection{Step 0: Characterize Good and Bad States}

When admitted to a hospital a patient goes through tests and examinations. Some of these are crucial to make a diagnosis for pneumonia (temperature, pain in the chest), while others are less important (demographic information). In addition, some of these test results are more useful for predicting the degree of disease progression. We refer to them as \textbf{significant tests}. 

To identify if a test result is good or bad for a patient, we consider all patients and all tests. A test result is considered good if it's associated with a patient who is discharged and bad if the test result is associated with a patient who died.
%patient is in a good or bad state we used a Decision Tree Classifier trained on the test results across 1419 patients and almost 400 thousand tests. In this decision tree, each non-leaf node denotes a test result, each branch corresponds to an outcome of the test, and each leaf node denotes a class prediction (see \ref{fig:DecisionTree}). 
We take into account only the last test results for each test type for each patient, i.e., the most recent results before a patient got discharged or died.

%Currently, the decision tree classifier performs with 69.72\% accuracy. Once our methodology has been refined, we may consider switching to a different classification model to improve classification accuracy - for example, Support Vector Machines (SVM), Neural network or Bayesian networks, or use ensemble of classifiers. This would reduce the variance of estimation errors and improve the overall classification accuracy similarly to the techniques that were used in decision support system for cardiovascular disease level prediction\cite{AptaSDSS}. We have started with decision trees because they are the easiest to interpret and hence check for reasonableness.

In subsequent sections, we use the median values of discharged patients and distance from those to determine improvement.



\iffalse
\begin{figure}[h]
\caption{A decision tree example for predicting patient's state. In this example, patient samples are classified into one of the two target classes (survives or dies) by testing the most recent test results before the patient is discharged or dies.}
% \centering
\includegraphics[width=16cm,scale=0.4]{figures/DecisionTreeOutcomes.pdf}
% \vspace{-100pt} % fix
\label{fig:DecisionTree}
\end{figure}
\fi



\subsection{Step 1: Compile Action-State Data}

The goal of step 1 is to be able to query what effect an action could have on a patient. This gives us the framework needed to make predictions about the effects of actions on future patients. To do this, we would require data points describing the effect an action had on patients. We represent these as triples consisting of the most recent value of the test before the action initiated, and the first value of the same test after the action is complete. A detailed example that illustrates how triples are formed is presented in Figure \ref{fig:triplesExample}.
%Each test would correspond to a set of triples, consisting of the value of the test pre-action (i.e. before the action), the test taken, and the value of the test post-action. For each action-test triple, we required the value from the last time the test was taken before the action, and the first time from after the action.
\begin{figure}[H]
% \centering
\includegraphics[width=16cm,scale=0.4]{figures/TriplesExample.png}
\caption{This figure illustrates how the triples are formed. On the left, measurements (in green) and actions (in red) are listed as they are performed consecutively one after another. On the right, formed triples are presented. Note that each action gets an entry for each test that was done before and after the action was taken. This way action Ibuprofen has two entries and records Temperature 1 and Bacterial Count 1 as values before the action. Values after the action represent the first value for each measurement after the action has finished, thus values Temperature 2 and Bacterial Count 2 are recorded for Ibuprofen. For action Antibiotic, similarly, 2 entries get recorded, where the latest test measurements before action started are Temperature 2 and Bacterial Count 1. Note that for values after Antibiotic are Temperature 4 and Bacterial Count 3 instead of Temperature 3 and Bacterial Count 2, which represent the first measurements done \em{after} the action was \em{finished}.}
% \vspace{-100pt} % fix
\label{fig:triplesExample}
\end{figure}

To do this, we constructed a table describing the triples. We look at every action done to the patient, and compile two temporary tables, one of pre-action test values and one of post-action test values. In both of these tables, an entry recorded the ID of the hospital admission in question, the IDs of the action and test the record was for, and the timestamp of when the action was taken. We then join the tables, matching on the hospital admission ID, action and test keys and the action timestamp, and the values returned by the test in both cases (pre-action and post-action). By doing so, we essentially have a table describing the effect of an action on the patient. For convenience, we also store the change in state variable value in its delta form if it was a numerical variable. From this table, we are able to query the triples required for the next step.

\subsection{Step 2: Identifying Median Healthy Test Results}\label{medians}

In order to determine whether the result of a given test is good or not, we require a notion of healthy and unhealthy results per test. To do this, we assume that the last measurement of a particular test taken from a patient who is about to be healthily discharged is a normal, healthy value for that test. We therefore take all these final measurements from discharged patients and, for each test, calculate the median value as well as the 90\% confidence interval around this median value. We store these medians and confidence intervals in a table, allowing us to check if a particular test result is normal (inside the confidence interval) or abnormal (outside the confidence interval). 

As described in Section \ref{Data encoding}, some test results have text values, so we cannot calculate median values for them. We have two methods for calculating the healthy values for such tests:

\begin{enumerate}
    \item
    Consider every value present in a patient at discharge as a healthy value. For example, if patients were discharged with Pain Types `None' and `Intermittent' we assume that both of these are healthy, and all other values for Pain Type will be considered abnormal. 
    \item
    Find the most common value for the test present in patients at discharge, and consider all other possible values as abnormal.
\end{enumerate}

We tested our system using the first approach.

\subsection{Step 3: Identify Abnormals} 

% Here is the strategy.
Test results that fall outside of normal ranges contribute lots of important information when deciding on the treatment of a patient. For instance, high temperature, cough and high bacterial count would indicate that a patient could have a flu. Our strategy for identifying such tests comes from detecting when tests deviate from the median values too much.

Assume we have a patient P who, at time t, has taken M tests and the most recent result values for each of these is given in the vector V. We denote this relation as \(testvalue(P(t)) = M.V\). 
Because some of these values may be normal (near the median of the values for discharged patients), and therefore do not contribute to our decision-making process for choosing actions, it is convenient to focus on the abnormal values.

%rephrase but first clarify the terminology
\(Abnormal(M.V) = M'.V'\) such that M' is the  set of measurements such that for every \(m\) in \(M - M', m.v\) is outside the 90\% confidence interval around the median healthy value.
%, as determined in Section \ref{medians}.
Thus \(Abnormal(M.V)\) consists of those measurement-value pairs of patient P that are  far from the median, because those are the ones that require attention.

\subsection{Step 4: Identify k Nearest Neighbours}
Next, we identify the states of patients immediately preceding an action or actions that improved their state from the dataset i.e the value came closer to the median values determined from good-state patients. We find those states that are ``k-close" to \(Abnormal(M.V)\) but may include other tests as well.

A state s is k-close to \(Abnormal(M.V)\) if for every test $m \in M$, the value of s on test m, denoted s[m], is one of the k nearest neighbors to \(Abnormal(M.V)[m]\) among the values of m.
If s is k-close to \(Abnormal(M.V)\), we say that s ``k-extends" \(Abnormal(M.V)\). If we do not find enough such states that ``k-extend" \(Abnormal(M.V)\), we consider the states that contain the largest number of k closest tests to \(Abnormal(M.V)[m]\).

Thus, we have a set of state-action pairs $(s,a)$ such that $s$ transitioned to a better state after the application of $a$ and possibly other actions and such that $s$  k-extends \(Abnormal(M.V)\). Call that set of pairs $P$ such that $P.actions$ are the actions, $P.states$ are the corresponding states and $P.states.tests$ are the tests performed in those states and $P.states.values$ are the values corresponding to those tests. %Call the set of states that k-extend \(Abnormal(M.V)\) the set $S$.


\subsection{Step 5: Identify Useful Actions} \label{identifyingActions}

Some actions may be more useful than others, depending on the state of the patient. We implemented and tested a number of strategies for identifying useful actions and discarding unnecessary ones. Two main approaches include calculating action utility and applying machine learning techniques to determine the importance of an action. Finally, we consider that if an action has already been done to a patient multiple times, it may have some recurring benefit and so we recommend taking it again.

\subsubsection{Action Utility}
An action's {\em utility} is measured as follows: the fraction of times with respect $P.states$ when the action is taken and state improved minus the fraction of times with respect to $P.states$ that the action was taken and state deteriorated. This yields a value between 1 and -1, with the upper and lower bounds indicating an action that was always useful and completely useless respectively. This notion of utility can be used to rank, and we can then decide on a cutoff above which we consider an action useful, yielding the set {\em useful actions}.

\subsubsection{Machine Learning Prioritization}
Once k-close states are determined, we attempt to construct a machine learning model for all actions represented in the k-close state-action pairs. We use this model on a patient's test result values to determine if that patient requires an action. For test results required by the SVM that the patient has not had done to them, we replace them with the mean values. 

\subsubsection{Repeat Strategy }
 If an action is repeatedly being taken by a patient, it is likely to be needed again. Therefore, we assign a utility of 1 to any action that appears in actions of k-nearest neighbours and the current patient has taken more than once.

\subsubsection{Strategies for Choosing Actions}

Our strategies for choosing actions to recommend are described as follows:

\begin{itemize}
    \item \textbf{Strategy KNN}. Recommend all the actions that are identified to be among the k-nearest neighbours.
    \item
    \textbf{Strategy +SVM}. Build a Support Vector Machine (SVM) for each action  identified by k-nearest neighbours and use it to determine whether the action should be done or not.
    %One plot of only the 'vanilla' version of each strat. Give random a clearly distinguished black line. DONE
    % Make the graphs zoomed, but start them from 0 DONE
    % Put the plots in one row of 3 DONE
    % Re-run w/ the same cutoff. Do the same thing as we do for seeing the result of repeat strat - a separate graph for comparing the effect of cutoff on each one DONE
    %Add graph for Effect of Neighborhood Size i.e. value of k DONE
    %^^This would make the point that finding the right K requires further study DONE
    %^^Fixed k and cutoff % DONE
    %^^Study repeat same k and cutoff% DONE
    %Base plot for each strategy uses k=50 and cut=-0.2 % OOPS 
    %Comparing the effect of k on Strategy 0 - take k=10,20,50,100, 150%
    %A graph for seeing the result of cutoff on Strategy 0 - take -1, -0.8, -0.6 up to 0.4%
    %Increase line thickness, use different glyph shapes, use highly differentiated colours, colorbrewer is a good website%
    %w.c. means word choice%
    \item
    \textbf{Strategy +Utility}. Since Strategy +SVM rarely identified actions to be taken we also tried automatically giving a utility score of 1 to any action recommended by the SVM, but not altering the utility score if action was not recommended by the model. We then discard actions with highly negative utility scores (utility $<$ $-0.4$)
    \item
    \textbf{Strategy Random Forest}. We proceed similarly to Strategy +Utility, but instead we use a Random Forest Classifier in place of an SVM.
\end{itemize}

On top of these strategies, it is also possible to perform Repeat Strategy (prioritizing actions that were previously done more than once).


\subsection{Methodology Example}
Consider a patient who enters with high temperature, normal blood pressure, a heavy cough, and brown hair. So, Vector M of measurements is (temperature, blood pressure, cough level, hair color) and V is (high, normal, heavy, brown). \(Abnormal(M.V)\) =  { (temperature, high), (cough, heavy) } because hair color is irrelevant to the patient's health state (we would know this because people with different hair color both die and are discharged) and blood pressure is normal (near the median for discharged patients). Given $Abnormal(M.V)$ we look for other patients that have high temperatures and heavy coughs and perhaps other values. For example, some other patient $q_1$ who has taken antibiotics may have { (temperature, high), (cough, heavy), (bacterial count, high)} and another $q_2$ who has taken an anti-viral may have { (temperature, high), (cough, heavy), (bacterial count, low)}. We employ one of strategies to decide whether antibiotics would be helpful to our patient.


% \subsection{Validation}



% To determine correctness we will (i) ask doctors for their validation of both extra measurements taken and actions taken (ii) perform leave-out tests.
% The leave-out tests will work as follows: 
% \begin{itemize}
%     \item 
%     Consider the measurements  done for patient $P$ at some time point, remove one of the measurements and see whether our approach would suggest doing that measurement.
%     \item
%     Consider the actions done for patient $P$ at some time point and see whether the decision tree inferred for $P$ using our method would suggest doing those actions.
% \end{itemize}

% It could be enough to measure  precision based on the top five recommendations or top k for some k.







%Next, we construct from our dataset of each other patient \(q\) a  row in a table E of all measure-values that extends \(M.V\), but have other measurement-values too. We find these value for times immediately before \(q\) undergoes an action that has helped patient \(q\) reach a better state. Each row of this table E, therefore, is of the form:\((testvalue(q(t')), a)\) where \(testvalue(q(t'))\) "extends" \(M.V\), and \(a\) is an action taken on \(q\) at time\( t`\), called a {\em critical timepoint} for \(q\), that improves \(q\)’s state. We define extending as follows:   \(measurementvalue(q(t'))\) extends\( M.V\)  if, for every measure\( m.v\) in \(Abnormal(M.V)\), \( measurementvalue(q(t'))[m]\) is close to \(v\). Value \(v' \) is  “close to"\( m.v \) for a given measurement \(m\) if, by sorting through all the v-values for measurement \(m\) and finding the k-nearest neighbors, \(v’ \) lies between the minimum and maximum returned values. Thus the notion of extending depends on some parameter k.

%Now, we take the set of all measures S defined as the union of the measures (tests) in all rows of table E. We next decide which measures in S - M to consider for patient P. We consider all the actions taken by patient q at time t' and call them A. We form a decision tree using the measures in S for each possible action a in A. We take the measures in M and see whether any actions $A_{mini}$ must be performed. For the remaining actions (in $A - A_{mini}$), we see which measures in S-M might be necessary to determine whether to undertake those actions or not.


% Using this table, we would then form another decision tree (if some patient has not had a measurement done, then we replace that value with the median of all values of that measurement). The decision tree tells us which measurements and which values are most important. We can therefore see which measurements need to be taken on our patient, and if any of those have not been taken, we do so.

\section{Results - Midpoint Testing}
To test the system on a given patient, we first select a point in time during the patient's stay, which we call a `midpoint'. `Midpoint' represents a point at a fraction of the duration of the stay (e.g. can be 1/2, 1/4th, 1/8th of the way through etc). All further testing was performed where `midpoint' represented 1/4 of the stay of a patient. We chose this point since it is enough time for a patient to have acquired some test results, but not enough time to be fully treated. At this point, we read all the patient's previous tests and infer actions that should be taken based on abnormal values out of those tests. We compare the list of recommended actions to all actions actually taken within a set time period following the testing point. The size of this time period is known as the `Testing Period Size'. The performance metric we used to evaluate the results include precision, recall and F-score. Precision is defined as a number of correctly recommended action over a number of all recommended action. 
\begin{equation}
Precision = \frac{number\;of\;correctly\;recommended\;actions}{number\;of\;all\;recommended\;actions}
\end{equation}
Recall is a number of correct recommended actions over number of all actions actually done to the patient after the `midpoint'.
\begin{equation}
Recall = \frac{number\;of\;correctly\;recommended\;actions}{number\;of\;all\;actions\;actually\;done\;to\;the\;patient\;within\;testing\;period}
\end{equation}
F-score is a harmonic mean of precision and recall.
\begin{equation}
F\;score = 2*\frac{precision*recall}{precision+recall}
\end{equation}
As per the strategies discussed in Section \ref{identifyingActions}, the following parameters are required to test the system against a given patient:

\begin{itemize}
    \item
    Which strategy is used for recommending actions
    \item
    Integer flag (1/0) to identify whether Repeat Strategy should be used 
    \item 
    The value of k for choosing k-close states
    \item
    The cutoff for action utility, below which we do not recommend that action
\end{itemize}

As a baseline, we compare the system's performance against one that chooses a random number of predictions to make, weighted on how many actions patients in the database had done to them. It then randomly samples that many actions, weighted by the frequency at which those actions were actually done. This list of randomly chosen actions is taken as a comparable set of action recommendations. To evaluate our system we performed Midpoint Testing utilizing combinations of the strategies presented above in comparison to the baseline. We perform the tests for Testing Period sizes of 1, 2, 3, 7, and 14 days. We also experiment with values of parameters such as cutoff and k. We summarize the results of our system in comparison to the random baseline in the following sections. All the experiments were conducted on a sample of 80\% of total pneumonia patients that had any actions done to them after the quarter point, which is $603\pm2$ patients. 


\subsection{Finding Optimal Neighborhood Size}

The number of neighbouring states ($K$) that we choose has a significant impact on our system's performance. This is because $K$ states will contain recommendations for up to $K$ actions, and by tuning $K$ we affect how broad or narrow the system is with its action recommendations. To find an appropriate value for $K$, we run Midpoint Testing on Strategy KNN for K=10, 20, 30, 50, 100, and 150. We calculate Precision, Recall and F-Score for each of these. This comparison is presented in Figure \ref{fig:comparingK}. As we can observe, the value of Precision falls with increasing $K$, whereas the value of Recall rises. The reason is that when we choose fewer neighbours, it is more likely for all chosen neighbours to be close to our patient. Therefore, the actions taken are more likely to be the same as the ones done to the tested patient. However, since the overall number of recommended actions will be low, recall is relatively low, and increases as we recommend more actions (i.e. as we increase $K$).

Based on this comparison, the value of $K$ was fixed to be 30 for all further testing. Despite not having the highest score on any of the three metrics, it has the third-highest F-Score. $K=10$ and $K=20$ achieve the highest F-Score, but the recall for these two values is too low to serve as a baseline. At the other extreme, $K\geq50$ perform worse than the baseline and therefore we discarded those options. To summarize, we choose $K=30$ to provide a balance of accuracy and sample size.

%Strategy KNN gives the highest possible recall our system can achieve, while by filtering actions, we can improve precision. Additionally, we would like to have more than 20 samples for the machine learning strategies. 

% Not sure where to put it, but here's the explanation for why the scores increase with an increased Testing Period Size
We also observe that for any Midpoint Test, the scores increase as the Testing Period Size increases. The reason is that the number of actions done in the 1 day immediately following the quarter point will naturally be smaller than the number of actions done in the two days immediately following the quarter point, and so on. Therefore, if we make a large set of predictions, a larger Testing Period Size is more likely to give higher Precision since more actions were done. This increase is more significant from Testing Period Sizes of 1,2, and 3 than to 7 and 14 since there are fewer patients who stayed for 7 or 14 days after the quarter point of their stay - therefore the number of actions actually done to them in a Testing Period Size of 7 and 14 may be identical. This effect is observed in all Midpoint Tests.
\begin{figure}[H]
\centering
\includegraphics[height=17.5cm]{figures/graphs/vertical/Comparing_K_highcontrast_vertical.png}
\caption{Comparing mean Precision, Recall and F-Scores for Midpoint Testing using Strategy KNN while varying the value of $K$. 
The X-axis shows the number of days following the`midpoint' for which we calculate Precision, Recall and F-Score. These scores range from 0 to 1 and are presented along the Y-axes.}
\label{fig:comparingK}
\end{figure}



\subsection{Finding Optimal Cutoff}

The effect of choosing a cutoff on Action Utility, as presented in Figure \ref{fig:comparingCutoff} are less obvious. We tested cutoff values by running Midpoint Testing for Strategy KNN, discarding actions below the cutoff. The test was run eight times with eight different values for the Cutoff from -1.0 and 0.4, at intervals of 0.2 each. The results are marginal, suggesting that a cutoff between -1.0 and -0.4 is optimal. For our tests, we set the cutoff to -0.4 because setting it lower does not effectively filter out actions.

\begin{figure}[H]
\centering
\includegraphics[height=17.5cm]{figures/graphs/vertical/Comparing_Cutoff_highcontrast.png}
\caption{Comparing utility score cutoff sizes on strategy KNN. The X-axis shows the number of days following the`midpoint' for which we calculate Precision, Recall, and F-score. These scores range from 0 to 1 and are presented along the Y-axes.}
\label{fig:comparingCutoff}
\end{figure}

\subsection{Comparing Strategies}
A comparison of Precision, Recall, and F-score for all strategies is presented in Figure \ref{fig:comparingStrategy}. For all experiments $K$ and cutoff were fixed at $30$ and $-0.4$ respectively. The most effective strategy was Strategy KNN, which recommended every action taken on a Nearest-Neighbour state. It achieved both the highest Recall and F-Score out of all four base strategies. High recall in this case indicates usefulness of our methodology for finding similar patients. +SVM and +Utility marginally achieved  highest Precision, however due to their low Recall did not achieve a good F-Score and are overall less useful than KNN. The Random Forest strategy also achieved an extremely high F-Score despite the lowest Precision (by a slim margin), due to its high Recall. In future work, it may be useful to continue to work on the Random Forest strategy through dimensionality reduction and other optimizations to the model. Currently, strategy Random Forest seems to be recommending nearly all actions. We can observe this in the fact that its recall is almost the same as that of Strategy KNN for every Testing Period Size. However, it has slightly lower precision than Strategy KNN, suggesting that we are not filtering out the incorrect actions.

To calculate the significance of our results, we take the F-score of our best-performing strategy (KNN) at a Testing Period Size of 3 days. We chose this period size since the improvement for period sizes larger than 3 is extremely small. We use the shuffling method between the recommendations of Strategy KNN and the random baseline, with 10,000 shuffles. 

To perform the shuffling test, we first find the difference in the F-scores of strategy KNN and the random baseline. We then get Strategy KNN's F-score and the baseline F-score for each patient. We iterate over the lists, swapping these values with probability 0.5. We calculate the means for the lists and store their difference. We do this 10,000 times. The p-value is the ratio of number of times the shuffled lists give a better mean difference than the original to the total number of shuffles.

For example, consider the following two patients defined by (patient, system f-score, baseline f-score):

(P1, 0.6 0.3)

(P2, 0.8, 0.5)

The difference in mean F-scores is 0.3. We swap system f-score and baseline f-score with 0.5 probability for each patient.
If only P1 has its values swapped, we now have:

(P1, 0.3 0.6)

(P2, 0.8, 0.5)

Now the difference would be 0 and we append this value to our list of differences. 
We repeat this experiment 10,000 times. Then we find a fraction of mean values in the differences list that exceed 0.3 (the original difference in means) by the number of shuffles. This would be our p-value.

Our highest-performing strategy yields a mean F-score of $0.417$ after three days, outperforming the mean F-score of the random sampler by $0.150$, with a p-value of $0.026$ (obtained through the shuffling strategy described), suggesting that the difference in performance is statistically significant.

\begin{figure}[H]
\centering
\includegraphics[height=17.5cm]{figures/graphs/vertical/Comparing_Strategy_highcontrast_vertical.png}
\caption{Comparing performance of strategies outlined in Section \ref{identifyingActions}. The X-axis shows the number of days following the `midpoint' for which we calculate Precision, Recall, and F-score. These scores range from 0 to 1 and are presented along the Y-axes.}
\label{fig:comparingStrategy}
\end{figure}

\subsection{Effect of Repeat Strategy}
Employing the Repeat Strategy of automatically recommending any action that appears in the KNN actions and has already been done to the patient more than once proved effective. For all experiments $K$ and cutoff were fixed at $30$ and $-0.4$ respectively. Repeat Strategy improves Precision, Recall, and F-score for most of the strategies and the increase in each score compared to the same strategy without Repeat Strategy is presented in Figure \ref{fig:evaluatingRepeat}. Since Strategy KNN already recommends all actions it is expected that it cannot improve when applying Repeat Strategy. The negligible improvement that can be observed on the graph is due to the fact that each experiment is run on randomized 80\% of the data set, thus we do not consider that Strategy KNN actually improved. Similarly, small improvement is also observed on Random Forest Strategy. This might be because Random Forest is already recommending a lot of actions and thus there are fewer additional actions to be recommended. On the other hand, once Repeat Strategy is employed on +SVM and +Utility, they outperform the baseline and their recall improves by at least 0.1 at each testing period. In addition, precision for these two strategies also improves slightly, which means that we are adding correct actions to our recommendations. To summarize, all strategies in this experiment perform better than the baseline on all three metrics. Repeat Strategy is therefore a useful tool for making action recommendations. 


\begin{figure}[H]
\centering
\includegraphics[height=17.5cm]{figures/graphs/vertical/Evaluating_Repeat_highcontrast_vertical.png}
\caption{Evaluating the effectiveness of Repeat Strategy across action prioritization strategies. The X-axis shows the number of days following the`midpoint' for which we calculate Precision, Recall, and F-score. The increase in each score compared to the same strategy without Repeat Strategy is plotted along the corresponding Y-axes. Individual scores for all strategies with Repeat Strategy are presented in Figure \ref{fig:repeatRawScores}. Individual scores for all strategies without Repeat Strategy are presented in Figure \ref{fig:comparingStrategy}.}
\label{fig:evaluatingRepeat}
\end{figure}


\section{Discussion - Qualitative Analysis}
While our system generally outperforms the baseline comparison system (viz. predict actions randomly weighted by their frequency), all strategies have large confidence intervals ranging from 0.03 to 0.6 for precision and 0.3 to 1 for recall. For comparison, the random baseline has even larger confidence intervals ranging from 0 to 0.8 for precision and from 0 to 1 for recall. Another limitation we faced is that the effective pneumonia database consists of only 969 patients, which is a small data set. 

\subsection{Finding Similar Patients}

The most effective method of improving recall when compared to random sampling was Strategy KNN. High recall in this case indicates usefulness of our strategy for finding similar patients. The drawback here was that we suggest a large number of actions. These included actions such as antibiotics (which would specifically help in curing pneumonia) as well as more general actions done to most patients (such as a saline drip). We could potentially use this list of recommendations to suggest treatments, or we may need to find an alternative method of picking the most relevant actions of this list, other than the methods attempted. 
 

\subsection{Identifying Actions using Utility Scores}
Calculating action Utility scores as described in Section \ref{identifyingActions} did not prove very effective in predicting which actions were taken. When picking actions that only had high priority (utility $>$ 0), our recall decreased without substantial gain in precision.
A potential explanation for why Utility scores were ineffective could be that they may pertain to the aggressiveness of a treatment. For example, an aggressive treatment that rapidly decreases a patient’s temperature would get a better utility score than one that has the same effect but over a longer period of time. We may therefore wish to see how the Utility scores are related to potency of the treatment. Also, currently, the utility score for a given action $a$ is a representation of improvements and deteriorations of all tests that appear in triples of action $a$ of all similar patients. This might take into account tests irrelevant to our patient and distort the Utility score. Therefore, it could be beneficial to take into account only abnormal value improvement/deterioration when calculating action utilities.


\subsection{Identifying Actions using Machine Learning methods}
Similarly, using Strategy +SVM to decide whether an action should be taken did not prove effective. Recall goes down without any substantial gain in precision. On the other hand, Random Forest augmented with Repeat Strategy performs comparably to Strategy KNN, which is the best strategy so far. Thus, we may consider improving the Random Forest strategy in future research. Currently, the models are trained on only a small number (30) of samples that represent K nearest neighbours (depends on the chosen K parameter described above). The number of features that the model is trained on is extremely high in comparison (500-600). Thus, it might be advisable to perform dimensionality reduction on the data before training the models in the future.

\subsection{Recommending Tests}
Going forward, we also wish to experiment with using our machine learning models to suggest tests/measurements that should be taken on a patient, and not only actions.
The logic could go as follows: we form a machine learning model (e.g. SVM or Random Forest) on patient P for each action $a$ in P.actions based on P.states.tests and P.states.values. This model could identify tests that are important for $a$. If some of these tests were not already done to a patient then they are potentially valuable when determining whether to take action $a$ or not.

\subsection{Number of recommended actions}
In addition to above conclusions, it was observed that a patient identified to have many abnormals generally has more actions taken. Currently, the number of neighbouring states (i.e. the k value) has a big effect on the number of actions taken. The number of abnormal measures is not taken into account - however, intuitively it seems likely that the number of abnormal values might directly correlate with how many actions are required to treat a patient. This leads us to consider a possible strategy where the number of actions should be positively related to abnormals. 

\subsection{Abstracting Actions}
Lastly, it was observed that often two different actions are really reflections of the same underlying idea. For example, two doctors recommending two different antibiotics to cure the same disease, or an IV drip of salt in one concentration vs salt in another concentration. This phenomenon drives precision of the system down, since our system often recommends both actions, while in fact only one was taken. Therefore, it could be useful to have a higher level notion of `action sets' that each action belongs to e.g. antibiotics, blood pressure medicine etc. We may then recommend an `action set', rather than a specific action.


% This step  first considers the question of which tests to do given that a patient is in some state s. We require some way to quantify how valuable a test will be. Intuitively, a test is potentially valuable for a patient P in some state, if the output of the test may change the action we take. A test is not valuable if we would take/not take that action regardless of the outcome of the test. 


\section{Summary and Impact}
This paper proposes a system that can recommend treatments for Intensive Care Unit (ICU) patients, given a set of previous measurements from which abnormal measurements can be identified. Our best performing strategy, KNN, yields a mean F-score of $0.417$ after three days, outperforming the mean F-score of the random sampler by $0.150$ (p-value $0.026$). With further work, this system can have its accuracy improved to more effectively recommend actions or action sets. It is also possible to implement and refine a methodology for recommending tests. In this case the system would be able to work on a continuous cycle of recommending tests, having their values inputted and recommending actions based on the abnormal values out of the new tests. This could potentially allow hospitals to be able to care for more patients without requiring more personnel. 

\section{Appendix}

Code for this project is available on request at \href{https://github.com/ShantanuBhatia/evidence-based-treatment/}{https://github.com/ShantanuBhatia/evidence-based-treatment}. Setup instructions and testing information are provided.

\begin{figure}[H]
\centering
\includegraphics[height=17.5cm]{figures/graphs/vertical/Evaluating_Repeat_Raw_Scores_highcontrast_vertical.png}
\caption{Evaluating the effectiveness of Repeat Strategy across action prioritization strategies. The X-axis shows the number of days following the`midpoint' for which we calculate Precision, Recall, and F-score. These scores range from 0 to 1 and are presented along the Y-axes.}
\label{fig:repeatRawScores}
\end{figure}


\begin{thebibliography}{99}
\printbibliography[heading=none]
\end{thebibliography}

\end{document}