Model for Prediction of Progression in Multiple Sclerosis

M sclerosis (MS), also known as demyelinating myelopathy, is a disease characterized by the appearance of demyelinating, neurodegenerative and chronic lesions of the central nervous system. MS is the second most common cause of disability in young adults and is associated with significant societal costs [1]. At this moment, no neurodegenerative or remyelinating therapies are available for clinical use and so the core of multiple sclerosis management lies in preventing episodic inflammation and relapserelated disability accrual. This fact together with the search for its causes, still unknown, is an active field of research. Because of disease’s effects on the central nervous system, it can present a series of symptoms that appear in the form of relapses or progress slowly over time, can result in reduced mobility and disability in the most severe cases. In general, five years after the appearance of the first symptoms, slightly more than 50% of patients have some kind of mild affectation, while 40% of patients show moderate affectation and less than 10% of patients die from the consequences of multiple sclerosis or its complications. Despite the rapid development of multiple sclerosis pharmacotherapy over recent years, there is no cure for this disease, but there are several multiple sclerosis disease-modifying therapies (known as DMTs). It is interesting to evaluate patients’ response to each DMT and which variables are good predictors for that response. Previous works evaluate and describe which clinical and biochemical variables predict the progress of a patient toward multiple sclerosis [2] [3], and study the long-term influence of relapsing-remitting MS (RR) [4]. But not only do traditional clinical factors influence the prognosis of this disease, [5] shows that quality of life levels provide additional prognostic information about MS disability. This reinforces the importance of incorporating other factors into the study. Therefore, accurate and timely detection of individual response to these DMTs is an essential requisite of efficient personalised multiple sclerosis therapy. Even though the prediction of individual disease course has now become feasible [6], [7], prediction of individual treatment response remains an area that needs further research. In this paper the efficacy of demographic, clinical and paraclinical variables of response to DMTs is evaluated and models are explored to determine those relevant in three stages of disease (Clinically Isolated Syndrome (CIS), Relapsing-Remitting MS (RRMS) and Secondary Progressive MS (SPMS)), without considering the other two stages (Primary Progressive MS (PPMS) and progressive relapsing MS (PRMS)). The international MSBase cohort has been analysed, and Machine Learning algorithms and Big Data techniques have been used to extract knowledge about this disease. Keywords


I. Introduction
M ULTIPLE sclerosis (MS), also known as demyelinating myelopathy, is a disease characterized by the appearance of demyelinating, neurodegenerative and chronic lesions of the central nervous system. MS is the second most common cause of disability in young adults and is associated with significant societal costs [1].
At this moment, no neurodegenerative or remyelinating therapies are available for clinical use and so the core of multiple sclerosis management lies in preventing episodic inflammation and relapserelated disability accrual. This fact together with the search for its causes, still unknown, is an active field of research.
Because of disease's effects on the central nervous system, it can present a series of symptoms that appear in the form of relapses or progress slowly over time, can result in reduced mobility and disability in the most severe cases. In general, five years after the appearance of the first symptoms, slightly more than 50% of patients have some kind of mild affectation, while 40% of patients show moderate affectation and less than 10% of patients die from the consequences of multiple sclerosis or its complications.
Despite the rapid development of multiple sclerosis pharmacotherapy over recent years, there is no cure for this disease, but there are several multiple sclerosis disease-modifying therapies (known as DMTs). It is interesting to evaluate patients' response to each DMT and which variables are good predictors for that response.
Previous works evaluate and describe which clinical and biochemical variables predict the progress of a patient toward multiple sclerosis [2] [3], and study the long-term influence of relapsing-remitting MS (RR) [4]. But not only do traditional clinical factors influence the prognosis of this disease, [5] shows that quality of life levels provide additional prognostic information about MS disability. This reinforces the importance of incorporating other factors into the study. Therefore, accurate and timely detection of individual response to these DMTs is an essential requisite of efficient personalised multiple sclerosis therapy. Even though the prediction of individual disease course has now become feasible [6], [7], prediction of individual treatment response remains an area that needs further research.
In this paper the efficacy of demographic, clinical and paraclinical variables of response to DMTs is evaluated and models are explored to determine those relevant in three stages of disease (Clinically Isolated Syndrome (CIS), Relapsing-Remitting MS (RRMS) and Secondary Progressive MS (SPMS)), without considering the other two stages (Primary Progressive MS (PPMS) and progressive relapsing MS (PRMS)). The international MSBase cohort has been analysed, and Machine Learning algorithms and Big Data techniques have been used to extract knowledge about this disease.

II. Proposed Methodology
A tool has been developed that is able to predict an individual treatment response by using the patient's clinical information. This tool serves as a consultation and aid in medical decision-making. In order to achieve this work, a multidisciplinary team has been formed with clinical experts and data scientists, who have worked together to validate the data used and the results obtained. This work provides an opportunity to study the evolution of multiple sclerosis in an international sample of considerable proportions.
This section is structured as follows: First, patient data collection is discussed. Second, a descriptive analysis of the sample is carried out as per the different variables of the patients, with a study of their different behaviour depending on different factors, and a global vision of the disease is acquired. Most relevant variables are used to predict the response to a treatment. A study is presented on how to quantify this response based on available data and to implement a prediction model using the most relevant variables with Machine Learning and Big Data techniques.

Data Collection
The real-world data set in the MSBase Neuro-Inmunology Registry 1 [8] was provided in plain text files directly from the international MSBase cohort. Every file contains anonymized clinical information about patients, compliant with the General Data Protection Regulation (G.D.P.R.), along with longitudinal data from their electronic medical records (EMRs).
Five databases were used as data sources. They stored 551,440 visits, 150,217 relapses, 235,902 magnetic resonances imaging (MRIs), 193,130 treatments and 48,186 patients from 64 different countries. All these records were in a structured format.
The raw data consisted of all the available data up to that moment, coming from clinical records, covering a period of time slightly longer than 50 years, from 1966 until May 2017.
A total of 185 variables were available in the raw data set. A treatment to prepare the variables was necessary due to the nature of the data and the disparate way in which doctors and countries filled in the variables. The selected variables (attributes or features) were reviewed by a team of clinical experts. As many as 92 new variables were built, and two variables extracted directly from the raw data, resulting in 94 explanatory variables to describe a patient.

Data Preparation and Validation
The extracted data are often incomplete, contain unnecessary or ambiguous information, suffer disruptions due to noise or pose other difficulties that affect the performance of the predictive models. Therefore, it is necessary to pre-process and validate them to avoid future inconveniences.
The process of extracting variables out of the patients' data is long and tedious and needs some collaboration from the clinical expert to validate them. The first step was the validation of each extracted variable with the medical team, thus obtaining those that are relevant and are usually filled in correctly by the medical community. In that process, we study how each variable is filled in, its degree of objectivity and the presence of absent values.
The inclusion criteria to enrol patients were: availability of the minimum dataset (i.e. patient gender, birth date, first symptom date and first stage of disease) and consistency between dates in their records and date of birth.
1 https://www.msbase.org/ Visits were included, but only those without relapses (i.e. there is no relapse in at least 30 days prior to the visit). This decision was adopted due to the large number of absent values in longitudinal data relating to Extended Disability Status Scale (EDSS) score, which measures the patient's disability [9], associated with each visit and with lesions in the MRIs. The clinical expert team consider EDSS information of great relevance.
Names of each Disease-Modifying Therapy (DMT) were coded with the main active principles corresponding to MS treatments, and any medication not specific to the disease or lacking a start date was discarded. Fourteen DMTs were studied, namely Aubagio Avonex, Betaferon, Extavia, Copaxone, Cyclophosphamide, Gilenya, Lemtrada, Novantrone, Plegridy, Rebif, Rituximab, Tecfidera. Tysabri. Betaferon and Extavia have the same active principle (Interferon beta-1b) and therefore they were grouped together for the study.
This data assessment used a process of data quality and generalizability that allowed identification of any incomplete, invalid or inconsistent entry.

Data Exploration
Some findings originated during data exploration should be highlighted.
In the study of the evolution of DMTs in patients following a CIS-RRMS-SPMS progression, different behaviours are observed whose start date is between two periods of time If several DMTs were supplied at the same time, the combination of these DMTs has been adopted. Fig. 1 (study from 1975 to 2005) presents a total of 1,687 patients with a maximum of six DMT changes. If a patient does not reach those six changes, the last DMT given was considered for the following changes. Fig. 2 (study from 2005 to 2017) presents 521 patients. In this case, the maximum number of DMT changes is seven. As before, if those seven changes are not reached, the last DMT provided was taken in the following changes.
In Fig. 1, 24.42% of patients have at least one DMT change. However, in Fig. 2 the percentage increases to 71.4%.  The most noticeable changes range from an old treatment to a new one. For instance, 39.6% of patients who start taking Avonex (TR2) and have a change, do so at a new DMT. The same occurs with Betaferon and Extavia (TR3), increasing the percentage to 46.5% of patients.
In addition, Fig. 3 shows average solar radiation per country. It is an example of demographic variables in the patient that are included in the model.

B. Prediction Model
A model was implemented to evaluate the response to fourteen DMTs using the demographic, clinical and paraclinical variables previously described.
MS behaviour was predicted, only patients whose course of the disease follows a CIS-RRMS-SPMS progression using the relapses stored in the clinical records. All information prior to the relapse, as well as the most recent treatment provided between the relapse and the time horizon to predict, was used. The case of not treating the patient is included as another predictable result.
As an additional requirement to those set during Data preparation described in subsection II.A, patient relapses were studied only if the following pre-relapse information exists: • An EDSS record in the absence of relapses (previous 30 days).
• If cerebral or spinal cord type MRIs is present, the variables number of lesions in T1 Gadolinium+ and number of lesions in T2 fields cannot be absent.

Prediction Outcomes
The response to a treatment is evaluated by the evolution of the patient's disability through EDSS. Therefore, this model forecast the progression of disability in two ranges, namely 1 and 2 years.
Progression of disability was defined as [10]: • Increase of 1,5 or more steps in EDSS if previous EDSS was 0.
• Increase of 1 or more EDSS steps greater than or equal to 1 and less than or equal to 5.5.
• Increase of 0,5 or more steps in EDSS greater than or equal to 6.
Only the progression events confirmed over greater than or equal to 6 months (with the confirmation EDSS recorded greater than 30 days following previous relapses and irrespective of treatment status at the time of confirmation).
Progression of disability in 1 year was defined and indicated whether the patient has progressed in the EDSS in a year. Progression of the disability in 2 years indicates whether the patient has progressed in the EDSS in two years, assuming that the patient remains at the same level of EDSS during the first year.
In this case, only relapses that have an EDSS record after the first or second year were taken into account, confirming that the patient had not progressed in their level of disability. Otherwise, there was no follow-up of the patient and it could not be confirmed/discarded that the event occurred.

Machine Learning Techniques
The target was to predict a discrete label: No progression or Progression in EDSS. In other words, the estimation of the probability of an event either occurring or not. These algorithms provide a score of the probability of the event occurrence. The interest was also to obtain a probability score associated with the non-occurrence of the event. Moreover, a threshold was set to decide this non-occurrence (i.e. our model predicts No progression if the provided score is above the threshold).

a) Feature Selection
A Random Forest model was implemented to predict the progression in the level of disability in response to treatment in1 and 2 years. It is able to provide the importance of variables for classification. The 14 most relevant variables chosen are shown in Table I. The most relevant variables for prediction are the following: • Gender is the patient gender, which is a factor type variable. Two binary variables were generated that indicate whether the gender is male or female. • Birth_date is date of birth of the patient. The age of the patient is used in the model. • Tab_rel_2 is the annualized relapses rate in last two years, which corresponds to the number of relapses in the last two years divided by two. • Num_rel is the total number of relapses. • Les_t1_gado is the number of lesions in the last RMI of the brain, spinal cord or cervical cord in T1 Gadolinium+ (relaxation time measured by MRI). If RMIs are not simultaneous, only the last RMI of the brain is taken into account. • Les_t2 is the estimated number of lesions in the last RMI of the brain, spinal cord or cervical cord in T2 (relaxation times measured by MRI). If RMIs were not simultaneous, only the last RMI of the brain is taken into account. • Current_dmt is the treatment currently prescribed. It is a factor type variable, with fourteen factors corresponding to treatments described. • Time_ms_dmt_ini is the time elapsed between the date of diagnosis and the start of the first prescribed DMT. • Time_current_dmt is the time elapsed between the start date of currently prescribed DMT and the date of last visit.

b) Classification Algorithm
To build the prediction model, the algorithm chosen was also Random Forest.

Random Forests
Random forests are ensemble learning methods that operate by constructing a multitude of small decision trees at training time and outputting the class that is the mode of the classes of the individual trees. Random forest is considered one of the best performing algorithms, especially for problems that have many explanatory variables [11].
The prediction model was trained using 70% of the train data, with the conditions and requirements previously explained. The remaining 30% was used to evaluate the accuracy of the model (test) in predicting the evolution of the patient for the treatment provided from the outbreak and compared to the actual evolution.

III. Results and Discussion
In this section, the evaluation of the model on the set of test patients is explained, considering the accuracy and validity of the results for different thresholds.

A. Results
In order to validate the model, the Receiver Operating Characteristic (ROC) curve for both years was first performed (Fig. 4), obtaining an area under the curve (AUC) of 0.8 and 0.82 respectively. In a ROC curve the true positive rate (Sensitivity) is plotted in function of the false positive rate (1-Specificity) for different cut-off points. Each point on the ROC curve represents a sensitivity/specificity pair corresponding to a particular decision threshold. A test with perfect discrimination (no overlap in the two distributions) has a ROC curve that passes through the upper left corner (100% sensitivity, 100% specificity). Therefore, the closer the ROC curve is to the upper left corner, the higher the overall accuracy of the test [12].
In addition, a measure of the error in terms of the patient's nonprogression was defined. That way, the probability the event nonoccurrence with a treatment.
For a given threshold, the error metric used (see eq. 1) indicates the proportion of patients who do not actually progress from the set of patients that the model predicts that would not progress. (1) A score above the established threshold ensures, with a probability equal to the accuracy of the model, that the patient will not progress in EDSS.
It was found that 76% of patients in the data set do not progress in the first year and in the second year that proportion is reduced to 67%.  As the threshold increases, the accuracy with which the model predicts that a patient will not progress increases considerably.

B. Functionalities
A web application called Model MS was built to support the specialist neurologist with the results of the model. This tool is one more component in the study process of a patient to decide what treatment to prescribe. Please note, however, that it is ultimately the doctor's decision to determine the treatment based on their experience and direct contact with the patient. Model MS is just a support tool. Fig. 6 shows the main screen of the tool, where the expert can: a) consult the minimum characteristics used for each patient (and previously described in Table I), b) search patients, c) upload patient data from an Excel file extracted directly from the platform for medical software systems ©iMed [13] or d) enter these minimum data manually. In this screen, it is also possible to change language and view a short description of the tool.

Output Data
Model MS presents three graphical and textual outputs, which are, respectively, a prediction of progression, evolution of the patient, and a description of the patient study that helps the expert.

a) Prediction of Progression
On the screen shown in Fig. 7, experts can view the progression forecast in EDSS from the last relapse. The predictive model uses the patient's current conditions (most relevant variables) and provides a score for each of fourteen treatments (described in subsection II.A), including the possibility of not treating the patient. The higher the score, the greater the likelihood that the patient will not progress if that treatment is prescribed. Fig. 7. Progression forecast. Score for each of the fourteen treatments that can be provided to patient. The higher the score, the greater the likelihood that the specific patient will not progress if that treatment is prescribed.

b) Evolution of the Patient
In Fig. 8 treatments prescribed to patients and evolution of course of them during MS duration are shown. The age of the patient when first symptom occurred, and MRIs are provided. The lower part shows the evolution of the patient's level of EDSS along with the visits and relapses.

c) Description of the Patient
In the screen shown in Fig. 9, a report is generated for the medical specialist, as a textual summary of what has been shown in the two previous screens. This report can be downloaded by the specialist.

IV. Conclusion and Future Work
In this paper a system has been designed with a prediction model that extracts implicit knowledge out of data.
This work, and the use of the Model MS tool, has several advantages: On one hand, it is possible to study the evolution of the disease from the different available variables (types of MS, geographical area, date of birth, gender...) and to identify several clinical factors relevant to its progression. Fourteen different DMTs and their effectiveness according to the different groups of patients analysed were studied.
On the other hand, a personalized prediction model for each MS patient, applying Machine Learning and Big Data techniques, has been developed. The resulting tool provides a probabilistic estimate of disease progression for different time horizons, and indicates the statistical conclusion of the probable impact of each treatment on the patient's evolution.
In addition, it has been verified from experimental results that not treating a patient is the worst option. The model always predicts a significantly lower score than any of the DMTs.
Finally, it has been verified that each patient presents a different evolution for each of treatments depending on their clinical state. There is no common behaviour for each DMT and it is normal that there are significant differences between the scores of the model in each one, but each patient has specific treatments that are better suited to their current clinical state. The best DMTs are those with the highest score, i.e. those in which it is most likely that the patient will not progress in EDSS.
The implemented tool has proved to be of interest and useful to neuroscientists in different countries. Model MS is being used in real time clinical practice by medical experts, to improve their results. A new line of work has been created to apply this methodology to many other branches of Medicine.