Conceptualizing the e-Learning Assessment Domain using an Ontology Network

—During the last year, approaches that use ontologies, the backbone of the Semantic Web technologies, for different purposes in the assessment domain of e-Learning have emerged. One of these purposes is the use of ontologies as a mean of providing a structure to guide the automated design of assessments. The most of the approaches that deal with this problem have proposed individual ontologies that model only a part of the assessment domain. The main contribution of this paper is an ontology network, called AONet, that conceptualizes the e-assessment domain with the aim of supporting the semi-automatic generation of it. The main advantage of this network is that it is enriched with rules for considering not only technical aspects of an assessment but also pedagogic.


I. INTRODUCTION
N the last decade use of the Semantic Web technologies as tools for generating, organizing and personalizing elearning content including e-assessment has attracted a great deal of attention [1], [2], [3], [4].Within the applications related to assessment, these technologies could be used for different purposes [5]: (1) to capture the structure of a domain, (2) to capture experts representation of a domain, (3) to encode and bind content to a domain structure, (4) to score knowledge map, (5) to package and deliver content at different grain sizes, (6) to be part of a recommender system, and (7) to provide a structure to guide the automated design of assessment.
In literature, different approaches that define an ontology as an structure to guide the automated design of assessment can be found [5], [6], [7].In [5] the authors have defined an ontology for supporting open questions generation whereas in [6] the authors only model simple choice questions.In [7], ontologies are used to generate individual problems examples for students that consist of a question and its solution.In spite of the advances done in this area, previous approaches have defined lightweight ontologies that only model the assessment domain from a technical viewpoint.
In order to e-Assessment be accepted by educators, a tool for supporting devising of valid and reliable assessments, from a pedagogical perspective, is needed.That means, it is required to establish an alignment of teaching, learning and assessment, and to define a mechanism for validating if the assessment covers all the learning objectives of a course and satisfies certain pedagogical principles [8].With the aim of solving this problem, two main challenges have to be addressed.On the one hand, it is necessary to link the different knowledge sources involved in e-Assessment: the subject domain, the assessment domain and the learning objects in which the assessment has to be based.On the other hand, a set of rules that model the pedagogical principles that an e-Assessment has to fulfill is needed.
The main contribution of this paper is an ontology network, called AONet, that formalizes the conceptualization of the knowledge related to assessments in e-learning environments considering technical and pedagogical aspects.The use of networked ontologies in the context of e-Learning has been addressed by other authors.In [9] the authors address the problem of specifying the semantics relationships between networked ontologies by defining an specification of these semantic relationships for the conceptualization of a Educational Recommender Systems.In contrast to this work, the contribution of this paper is the conceptualization of the assessment in e-Learning.
The present paper is organized as follow.Section 2 defines the main concepts around the approach of this paper.Section 3 presents the main components of the AONet ontology network.Section 4 discusses an example of the AONet population.Finally, Section 5 is devoted to the conclusions and future work.

A. Ontology Definition
An ontology gives an explicit definition of the shared conceptualization of a certain domain [10].Since ontology were used for different purposes in different discipline, several definition were built.Then, it is necessary to clarify what we have in mind when we talk about ontology.The definition used in this paper is based on [11].
From a pragmatic perspective, an ontology can be defined as a representational artifact based on four kinds of modeling components: concepts, roles, restrictions and individuals.In ontological community, ontologies can be classified as lightweight or heavyweight.A lightweight ontology is an ontology simply based on a hierarchy of concepts and a hierarchy of relations whereas a heavyweight ontology is a lightweight ontology enriched with rules used to fix the semantic interpretation of concepts and relations [10].
The component that differentiates an ontology is the set of rules.This set has to be expressed in an appropriate logical language.Considering that the OWL language is the standard for implementing an ontology and this is not always enough to do some deduction, then it is needed to combine OWL with other representation formalism as rules.One of the integration approaches is the Semantic Web Rule Language (SWRL), which provides the ability to express Horn-like rules in terms of OWL concepts [12].
In order to extract information from OWL ontologies a query language is needed.The most powerful language is SQWRL, which is based on the SWRL rule language and uses SWRL's strong semantic foundation as its formal underpinning.It also contains novel set operators that can be used to perform closure operations to allow limited forms of negation as fail-true, counting, and aggregation [13].

B. Ontology Network
An ontology network is a set of ontologies related together via a variety of different relationships such as mapping, modularization, version, and dependency.The elements of this set are called Networked Ontologies [14].
An ontology network differs from a set of interconnected individual ontologies in the relations among ontologies since in a ontology network the meta-relationships among the networked ontologies are explicitly expressed [9].There are some models that cover both the syntactic and semantic aspects of dealing with ontology relationships in networked ontologies.In the DOOR (Descriptive Ontology of Ontology Relations) ontology, general relations between ontologies, such as includedIn, equivalentTo, similarTo, and versioning were defined by using ontological primitives and rules [14].
Concerning a support for implementing and management ontology networks, the NeOn Project can be mentioned (http://www.neon-project.org).NeOn has developed an open service-centered reference architecture for managing the complete life cycle of networked ontologies and metadata.This architecture is realized through the NeOn Toolkit and complemented by the NeOn methodology, which is a scenario-based methodology that supports the collaborative aspects of ontology development and reuse [15].
From a model integration point of view, within an ontology network each ontology conceptualizes a specific domain and plays a particular role.Then, the main advantage of using an ontology network is the conceptualization of a given domain in a modular way.The networked ontology is small enough to be understandable by any person and its maintenance is easy.In addition, several ontology designers could work on different networked ontologies concurrently.

C. The Assessment Domain
Assessment is an indispensable part of teaching and learning.Essentially, it is assessment that reinforces the learning approach a student adopts.If a student is often tested on higher-order thinking skills, they are likely to adopt the desirable deep holistic approach to e-Learning.On the contrary, if students are tested on lower-order thinking skills, they would probably be encouraged to practice the undesirable surface atomistic approach to learning [16].An assessment can be considered as difficult to be realized within a distance learning phase.
Assessment can be classified in formal, informal and semi-formal assessment, depending on the formality and structure of assessment instruments [17].Thereby the formal assessments are structured: there is a place and a time setting where they are carried out.There are different types of formal assessment: simple choice, multiple choice, correspondence, conceptual maps and performance evaluation among others.The semi-formal assessments are homework and tasks that the student makes during lesson day and continue out of it.These types of assessments are for example reading comprehension, mathematical problems, trials, projects development, programming, conclusion development, outcome analysis among other.The informal assessments are not structured at all.They consist of quizzes and activities observations that the teacher makes during class and consume a few minutes.Some instruments that are used for systematize these types of assessments are: class daily (class journal), control list, anecdotic annotations among other.
It is considered that an assessment is composed of reactive.When teacher elaborates a reactive in order to make an assessment, uses the Bloom taxonomy [18].This taxonomy is used to classify the course or programs goals as function of six level of complexity: --First, click on the View menu and choose Print Layout.Knowledge: in this level teacher wants to evaluate the concept memorized by students, for example question about concepts.
--Comprehension: teacher wants to evaluate if the student understands the semantic relation of information taught.For example, conceptual maps.
--Application: teacher wants to evaluate if student can use the information taught to solve practical problems, for example mathematic problem.
--Analysis: teacher wants to evaluate the structure of knowledge, for example, outcome analysis.
--Synthesis: teacher wants to evaluate if student can elaborate original approaches base on concepts taught, for example trial.
--Evaluation: teacher wants to evaluate if the student can make a value judgment on topics taught, for example, conclusion development.

III. THE AONET ONTOLOGY NETWORK
With the aim of developing the AONet ontology network (Figure 1), the guidelines defined by NeOn Methodology were followed [15].All of the ontologies defined in the AONet are implemented in OWL DL 1.0.Following, each of the ontology that composes the AONet is described The Educational Domain Specification Ontology comprises concepts and relations defined in the knowledge domain that is evaluated.As can be noted, its structure and content depends on each particular domain.
The Educational Resource Specification Ontology comprises the educational resources used by educator in the teaching-learning process (TL).Some standards emerge to overcome the formalization of educational resources which are constantly evolving.In most cases, the use of learning object (LO) definition and its description by LOM [19] is the common denominator.In this way, it is possible to optimize the educational resource development process.This ontology is related with Educational Domain Specification ontology throughout use relationship.This relation identifies the connection between educational resources and concepts belonging to the specific domain.That is to say, an educational resource is developed in order to overcome different concepts, relations and definitions about to a domain topic.A LO metadata instance describes relevant characteristics of an educational resource, with the aims of facilitate the search, acquisition, interchange and evaluation of a resource by teacher, students and software systems.For this reason, we add to the ontology network the LOnto ontology built by Romero and Godoy (2010), which conceptualizes the semantic definition of LO based on LOM IEEE 1484.12.1 standard [18].Then, the Educational Resource Specification ontology is related with LOnto through isSchemaFor relationship.The LOnto ontology is described in the next sub-section.
Assessments are part of the educational resources involved in the TL process when teacher wants to evaluate the concepts and skills acquired by students.In this context, the ontology network has the Assessment ontology which is related with Educational Resource Specification ontology through is-a relationship.In the same way, this ontology is related with Educational Domain Specification ontology through the evaluate relationship.These relations describe that an assessment is used to evaluate the results of the TL process about the Knowledge Domain.
There are different instruments to evaluate, which are modeled by the Assessment Instrument ontology.These instruments are used by teacher to generate an assessment.For instance an instrument is a True/False question, a conceptual map, an exercise, an essay activity among other.Then, the Assessment ontology has the use relationship with Assessment Instrument ontology.
The next sub-sections describe in detail the networked ontologies proposed in this paper.

A. The Assessment Ontology
The Assessment ontology (Figure 2) is the core of the AONet ontology network.This ontology conceptualizes the fact that an Assessment is an Educational Resource that is described by the LOM metadata (defined in the LOnto ontology).Each Assessment is composed by Activity.An Activity is a motto or exercise that evaluates a particular domain topic and it is composed by one or more Reactive which is an item that uses an Instrument (defined in the AssessmentInstrument ontology).
The objective of an assessment is to show that the learner has achieved competency in the topics of the unit or course being evaluated.These topics are conceptualizes in the Educational Domain Specification ontology.This ontology is dependant of the course and how it could be built is out of the scope of this paper.

B. LOnto Ontology
The LOnto ontology is based on the IEEE Standard for Object Metadata LOM.This ontology was built by performing the activities defined in the Methontology methodogy.A deeper description of the LOnto ontology can be found in [2].
The LOnto ontology is defined around the concept of LOMv1.0schema which is the superclass of all the elements and data types of the LOM schema.In the upper level LOM has nine metadata categories: --General: general information to describe LO as for instance title, keywords, abstract among other.
--Lifecycle: life cycle characteristics of a LO and revision.
--Meta-Metadata: information about the metadata instances.
--Technical: characteristics and technical requirement of a LO.
--Educational: characteristics of the LO relevant to the TL process.
--Rights: copy rights properties --Relation: characteristics that relate the LO described and other instances.
--Annotation: comments about LO in educational environments, and information about when and who develop its content.
--Classification: describes a LO related to a particular classification system (taxonomy).
For each metadata category above mentioned it has been defined in the LOnto ontology a class that extends LOMv1.0schemadepicting the aim of the metadata in this category.Classes are specialized in subclasses representing each particular element.Figure 3 shows a part of the LOnto ontology.As can be seen, there are nine subclasses of LOMv1.0schema:Technical_Metadata, Lifecycle, Meta-Metadata, Educational, Right, Annotation and General_Metadata.So, General_Metadata has two subclasses Title and General.Note that standard LOM describes a taxonomy of metadata for LO while LOnto not only takes into account this taxonomy but also add relation among elements and restriction rules.

C. AssessmentInstrument Ontology
The Assessmentinstrument ontology models different instruments that could be used in an assessment depending on the evaluation technique implemented.An assessment instrument is the physical support that is used to collect the information about the expected learning of students.This ontology is shown in Figure 4

D. Rules for determining the assessment quality.
According with [19] there are some pedagogical recommendations that teachers need to take into account in the development of assessment.If these guides are followed by teachers, we can say that the assessment is valid in a pedagogical sense.In this work, these recommendations were used in order to define rules to express the restrictions in the generation of valid assessment.Considering that Multiple and Simple Choice are the most used instruments, we use them in this paper to illustrate the rules.From a pedagogical perspective, it is recommended that there is always a right option.It is recommended also that this type of activities do not include options such as "none of them" or "all of them".In general, items should be belonging to the context of content area being assessed in a clear and simple way and preferably written in the affirmative mode.The distractors should appear as attractive as possible to the uninformed student.Table I shows the pedagogical rules that have been taken into account.The first column describes the rule in a colloquial language.Second column shows the fist-order logic description of such rules.Note that in using First-order logic we consider reification of concepts such as:   We have defined logical rules for representing each restriction above mentioned.Then, these rules were implemented in SWRL and SQWRL as shown next.

V. CONCLUSIONS AND FUTURE WORK
This work has shown a preliminary ontology network which purpose is to conceptualize the assessment domain in a TL process.The modularization that this network provides allows us concentrate the attention on a particular domain and incrementally build a more general model relating different ontologies.The concepts related with assessment domain were presented.Mainly, this work focused on describing the ontology network that models the different areas related to assessment in an educational context taking into account not only technical aspects but also pedagogical one.The LOnto ontology conceptualizes not only the metadata proposed by IEEE standard but also the relations and restriction among metadata that are not present in the standard, giving as result an improvement in the use of such standard.The Assessment ontology represents the main concepts found in an assessment domain, giving in a different ontology the instruments used to develop an assessment.In this way, we can consider on the one hand, the way in which to develop an assessment and on the other hand, the relation that this assessment has with students, teachers and educational program.Through ontology network it is possible to add new ontology and relates it with the existing one.The SWRL rules to determine the validity of a given assessment were presented.These rules are based on pedagogical criteria enabling assessment to be considered by educators in an e-learning process.In this first approach, we focus on multiple and simple choice activities due to they are the most popular activities used by educators in elearning.
Finally, an example of the ontology network population by using an Artificial Intelligence assessment was discussed.In the future, we intend to acquire additional validation assessments for a broad evaluation and refinement of the ontology.
We are working on improvement of the ontology network adding new concepts and relation.In turn, we are developing test using different assessments provided from different knowledge domain.In addition, we are working on developing a tool for supporting an assessment generation by using the ontology network presented in this paper.

Fig. 3 .
Fig. 3.An excerpt of the ontologies that compose the network.

Fig. 5 .Fig. 6 .Fig. 7 .
Fig. 5.The Artificial Intelligence Assessment.www.ai-class Figure 6 shows the result to instantiate the ontology network in order to represent the artificial intelligence assessment.Note that instances have a prefix that identifies the ontology they belong.The asse:ExamIntroductionToAI instance represents the assessment, it has two activities: asse:SearchActivity and asse:MachingLearningActivity instances and it has lonto:IntroductionToAITitle instance associated by the isSchemaFor relationship.Each activity evaluate a domain topic as it is shown with the relations between asse:SearchActivity and dom:Search instances and between asse:MachingLearningActivity and dom:MatchinLearning instances.

Fig. 8 .
Fig. 8. Maching Learning activity instantiation Concept represents classes of objects.Roles describe binary relations among concepts; hence they also allow the description of properties of concepts.Restrictions are used to express properties of roles, i.e. cardinality.Individuals represent instances of classes, i.e. objects.Additionally, it is possible to use axioms and rules to infer new information.Axioms are logical sentences always true that express the properties of model paradigm.Rules are logical sentences that express characteristics of the domain, i.e. business rules.Formally, . The main concept is Instrument.There are two types of instruments: FormalInstrument and SemiformalInstrument representing formal and semiformal techniques respectively.As semiformalInstrument, we have considered two type of it: SimpleInstrument such as Exercises, ConceptualMap and Essays, and CompositeInstrument as portfolios that consist of a collection of SimpleInstrument elements that help recording learning process and students' progress.As FormalInstrument we considered two classifications: EssayActivity, where students have to elaborate the answer and ObjectiveActivity, where students have to identify the correct answer.EssayActivity, is specialized in two subconcepts: RestrictedEssay and UnrestrictedEssay.ObjectiveActivity is one of the most used by professor because it eliminates the subjectivity in the rating, even when it has an additional complexity to develop it.Objective Activity has three sub-concepts: Choice, Correspondence and Completition.Choice has Option associated.The concept Option is specialized in two sub-concepts: Distractor and TrueOption.Distractor are items that are not correct and TrueOption is the correct item.The concept Choice is specialized in: SimpleChoice contains only one correct option and MultipleChoice can have more than one correct option.In both cases, Option can only have Boolean answer associated.Finally the concept Answer can be of different types: TrueFalse, Numeric, Text and Relation.