BROAD-RSI – educational recommender system using social networks interactions and linked data

Abstract

There are several educational resources distributed in different repositories that address to a wide range of subjects and different educational goals. The proper choice of these educational resources is a challenge. Recommendation systems may help users in this task. In order to generate personalized recommendations, it is important to identify information that will help to define user profile and assist in identifying his/her interests. The constant and ever-increasing use of social networks allows the identification of different information about profile, interests, preferences, style and behavior from the spontaneous interaction. This paper presents an infrastructure able to extract users’ profile and educational context, from the Facebook social network and recommend educational resources. The proposal is supported by Information Extraction Techniques and Semantic Web technologies for extraction, enrichment and definition of user’s profile and interests. The recommendation approach is based on learning objects repositories, linked data and video repositories. It takes advantage of user’s spent time at the web. The proposal evaluation was made from the development of a prototype, three proofs of concept and a case study. The evaluation showed users’ acceptance of extracted information about their educational interests, automatically generated from social network and enriched to find implicit interests. It was also validated the possibility of people recommendation, enabling the establishment of interest network, based on a specific subject, showing good partners to study and research.

1 Introduction

Easy access to computing devices such as computers, tablets and smartphones, along with the growing use of Internet have brought continuous and fast changes in people’s behavior in relation to access, search and dissemination of information. This is a multidimensional structural change in people’s life based on information and communication technologies.

Education domain has been influenced by this technological revolution. Computational tools are increasingly present in information access process, both by conducting formal classes supported by technologies, or in day by day people’s life searching to non-formal educational content (blogs, video channels, free classes, social networks, and so on). There are many educational resources distributed in different repositories that address to a wide range of content and subjects that can supply information to individuals or groups in the Web.

The proper choice of Web educational resources is a users’ challenge. Some technological tools contribute to educational resources location and selection, such as repositories of Learning Object [1, 2] and Recommender Systems of educational resources [3].

A Recommender System (RS) plays a key role in helping users to find educational resources relevant and pertinent to their profiles and context. So, it is necessary to identify information that helps user’s profile definition and in identifying requests and interests.

Constant and intense virtual interaction highlights many details about user’s profile, interests, preferences, contacts and behavior due to the spontaneous relations that take place in these systems. In formal education the virtual environments include Virtual Learning Environments (VLE) [4], Virtual Educational Systems and those that support massive courses – MOOC [5]. In a larger context are the social networks [6] such as Twitter, LinkedIn, Facebook, Instagram, among others.

All these educational systems can contribute to extract features of user profile and context. However, usually these systems are restricted to an educational institution or are specific for a users’ group. On the other hand, social networks do have not such restrictions but their wide use and scope produce a large volume of data, beyond possibilities of interaction among users and facilities in information diffusion.

For Junco [7], the most popular social network among college students is Facebook. According to a survey by QS Quacquarelli Symonds Ltd. [8], between 2012 and 2013, with 918 answers from 26 countries spread over Europe, Asia, Latin America, North America and Africa, the use of Facebook among students varied from 78% and 96% in these regions, being Latin America the region with more access to Facebook.

The same survey showed that about 30% of participants in Latin America and Africa spend five or more hours on social networks. If we add this number to the percentage of people who spend between 3 and 5 h we reach 78% in Latin America and 54% in Africa. The value is a little lower in regions such as North America, Europe and Asia. However, percentage of people who use social networks for more than 5 h a day remains between 24% and 26%. So, if we add these values to those who use social networks between 3 and 5 h, the number reaches 64% in North America and 50% in Europe and Asia.

According to [9] the use of social networks as educational tools is a trend. In addition, there are studies that show the feasibility and importance of using social networks both in formal and non-formal learning [10], in particular Facebook [11]. Moreover, [12] show that the use of Facebook data can significantly improve recommender systems results. They affirmed that, when using Facebook data, recommendation results are at least equally as accurate as results obtained from user ratings, but they did not address about educational recommendation.

For all these reasons, it is very motivating to explore the use of spontaneously generated information on social networks to help searching and selection processes of educational resources in an individualized and personalized proposal. It is also motivating the possibility of using social networks to offer educational resources that contribute to user scholar career, by exploring as long as they remain connected to these environments. We can also explore the power of information diffusion that social networks have, which may contribute to the wide distribution of educational resources. Besides its popularity, Facebook is one of the social networks that allows the extraction of users’ information, through a technological support, where developers can integrate their systems in a real context.

An additional motivation is to improve researches of Knowledge Engineering Research Group (NEnC) related to BROAD Project [13,14,15,16,17]. This project includes researches related to software technologies, such as ontology, semantic web services, agents, software ecosystems and recommender systems in Education domain. In this work, our architecture includes, besides the BROAD Learning Objects (LO) Repository, user’s profile features and educational context, through the semi-automatic information acquisition provided by Social Networks, enhancing personalized educational resources recommendation. In addition, we evaluated the feasibility of recommending educational resources from linked data repositories, comparing its adherence in relation to learning object repositories [17]. This paper extends NEnC research evaluating: profile features, extracted and discovered interests; recommendations from multiple data repositories; recommendation sharing by users; and others.

In this context arises the research question: “The use of users’ profile and context information, that are not necessarily in a formal educational environment, with daily automatic data update, makes possible to recommend educational resources in an individualized and personalized way?”. The main goal of this paper is to propose BROAD-RSI (Recommender System Based on Social Interactions), making possible to extract and exploit information available on social networks to identify features of user’s profile and context, and make individualized and personalized recommendations of educational resources. We present three recommendation approaches: based on learning objects repositories; based on linked data sets; and video recommendation based on YouTube.

This paper is organized as follows. Section 2 presents the theoretical basis and related works. Section 3 describes BROAD-RSI infrastructure, technologies solution, requirements, data workflow and recommendation approaches. Section 4 presents the evaluation process, through three Proofs of Concept and a Case Study. In section 5 are the final remarks.

2 Recommender Systems in Education

According to [18], a recommender systems can be “any system that produces individualized recommendations as output or has the effect of guiding the user in a personalized way to interesting or useful objects in a large space of possible options”. This type of system basically needs three components to provide recommendations: (1) context data, i.e., information that the system has before recommendation process begins; (2) input data, which is user’s information so that system makes a recommendation, and (3) algorithm used in recommender process that operates with context and input data, to provide user’s recommendations.

Recommender Systems use some techniques that allow us to choose the most appropriate content to a particular user. Burke [18] categorizes recommendation techniques in five sets: Collaborative Filtering (CF); Content-based Filtering (CBF); demographic; utility-based; and knowledge-based. According to [19], content-based and collaborative filtering techniques can be used in a combined way, it is called hybrid recommendation.

According to [3] Recommender Systems can be used in education to filter educational resources based on student individual or students group profiles in a class performance. Thus, these systems can help teachers in the selection of digital resources for a specific class and on students monitoring, since the concern to identify and treat different profiles is partially or even completely solved with customization provided by these systems.

Some other examples of educational tasks supported by Recommender Systems are [20]: recommend items to help student while he/she is performing a task [21, 22]; discover and display new items of interest [23]; adapt content and user interface [24]; recommend items in a relevant sequence [25]; provide educational resources to users as they browse on other sites [21, 26, 27]; find good pairs to work together [28,29,30]; recommend people with similar interests, preferences, and learning styles [28].

According to [31] the Information Extraction purpose is to identify predefined concepts set in a particular knowledge area, ignoring other irrelevant information, i.e., deriving structured information from unstructured text. The main features that may be found in an information extraction technique are [32, 33]: (i) necessity to get information from a data source typically composed by textual or images content, (ii) inability to obtain the humans’ answer due to large volume of data; (iii) lacking of formal structure data like those from a database which does not allow a direct query made by computers, for example.

The use of Semantic Web techniques and standards in education have brought improvements in some aspects, such as customized content and educational resources for students, educational resources recommendation, and data collection regarding the interaction of students with educational environments [34, 35].

In education, [36] points out that research on Semantic Web are focused on the following topics: building ontologies for e-learning; use of ontologies and Semantic Web standards for structuring, representing, indexing of shared and interoperated educational resources; use of ontologies and Semantic Web standards to support intelligent e-learning systems and use of context based on Semantic Web for adaptation and personalization of e-learning applications. BROAD Project architecture has a semantic database, composed of ontologies [37, 38]. These ontologies enrich our researches and are part of the recommendation system architecture. In educational context, ontologies can be applied to solve a variety of complex problems such as knowledge representation in intelligent systems, knowledge sharing, application reuse, annotation and search for learning objects, personalization of learning content, among many other challenges [36].

One of the main objectives of Linked Data is to extend the traditional Web, where HTML documents are interconnected via hyperlinks, to a Web with data that can be directly connected, without any application intervention. This web extension is also known as Web Data [39].

The concept of Linked Data has been exploited in some Education projects. [40] explains that there is potential for the use of linked data in Education because of the open educational resources produced by many universities:

“... while the content of open educational resources is by definition accessible and reusable, publishing the corresponding metadata as Linked Data can make the content of different repositories more discoverable, accessible and connectable. The use of common vocabularies to describe the topics is a way to make the content of these repositories globally addressable, and to retrieve resources by their relevance, without having to consider their origin.” [40].

In addition to publish it is important to emphasize the power of using data available through linked data in educational systems. The Talis Aspire Community Edition application [41] is a recommender system that uses the Linked Data approach. It allows UK university professors to create and manage lists of books online for their classes. These lists are arranged so that the metadata used are standardized and provided as linked data and similar items are linked to each other. After organizing and representing the data, the application is able to find similar items to those posted by user.

One application available on Facebook, which brings benefits from the possible connections with Linked Data is “Course Profile” of Open University. It uses the courses catalog as a source of information. This catalog was stored using the Linked Data approach and allows a user to connect to other users. So, it indicates whether they have attended, are attending or are interested in attending a particular course. With people’s indication it is also possible to suggest other activities or related courses.

2.1 Related works

In the context of this research, we selected recommender systems from education domain. Different approaches are used in context-sensitive Recommender Systems. Techniques used to define or enrich the user profile (information extraction through the use of the systems, interaction with the content, similarity between profiles, and so on; the recommendation technique (content-based collaborative filtering, hybrid, based on semantic rules, and others); and the recommendation target learning objects, educational environments, websites, courses, among others) define the mandatory differences among these systems.

A context-based architecture to recommend resources in Personal Learning Environments (PLE) is described by [26]. It focuses on the recommendation of independent services (Facebook, YouTube, personal calendar and the IEEE Xplore digital library) according to the user’s interest. The recommendation is based on the contextual information of services and students. The services context is captured through virtual sensors, and user context considers the information from his/her profile and feedback from previous recommendations. The user’s context information is categorized into personal (static) and preferences (dynamic).

In [28] is presented a recommendation proposal based on evaluation of educational content on a social network where students share with each other their ratings for resources, including books, educational materials, and so on. When the student needs a recommendation on a particular content, he/she sends a query to his/her immediate and distant friends. Immediate friends are those directly connected (one jump between them), while distant friends are linked by two or more jumps. Students, who previously used the resource, answered the query with their ratings. Based on the answers and the ratings history, the score for the recommendation is then calculated.

In [42] is described an educational resources recommender system that considers the context elements of the student using ontologies to represent it. There are two contextual information types: static context and dynamic context. In the static context, they consider personal information (e.g. name, experience, nationality) and information relating to the student’s educational environment, which generally are not subject to constant change. On the other hand, in dynamic context, information related to the student’s preferences and interests are considered relevant, and may change during the educational process.

An architecture is proposed by [43], that allows multiple educational content recommendations. The students are categorized by the teacher in relation to their knowledge level in a particular subject (basic, medium or advanced). The first recommendation is based on teacher suggestion, which defines the most appropriate content for each level. The recommendation engine will consider a range of information such as user profile, context information and quality of experience (QoE) to compose a tailored recommendation.

Through the analysis of these works we can identify difficulties as well as adopted solutions. The Recommender System presented in this paper goes forward and brings innovation comparing to [26, 28, 42, 43] as it extracts information spontaneously generated on social networks.

A recommender system architecture named Protus based on tags for e-learning environments is proposed by [44]. Protus is a mentoring system designed to help students in learning programming languages fundamentals. The main key is to recommend useful and interesting educational resources for distance learners, based on their activities (such as sequential patterns, pages visited, test results and grades), experiences, preferences, learning objectives and other significant attributes. The proposal uses a student model to recognize the goals of other students (learning style) and to filter learning objects that have appropriate content for them.

In [27] is presented a context-sensitive recommendation proposal to the development of an adaptive ubiquitous educational system (u-leaning). The goal is to allow students to build themselves managed and personalized environments. The appropriate educational resource is recommended to the student’s learning environment considering his context, and by using the wireless transmission and detection through sensors of learning goals in real situations.

A proposal that seeks to model the context using the extraction of exchanged messages between students on social network groups is presented by [21]. The context is used to improve the relevance of the document retrieval on the Web, aiming to support collaborative learning. The authors intent to make context-sensitive recovery information (using messages for context modeling), thus offering contextualized search results. The authors proposed to capture the context through enriched messages using open data to improve users’ queries, providing more appropriate content from the Web.

[45] present a recommendation technique based on implicit construction of user profile by observing the values of metadata from accessed content. The proposed recommendation technique uses collaborative filtering combined with a clustering technique that groups users with similar interests.

The possibility of sending recommendations in the social network, allows user to access educational content and also share it by suggesting it to other users. The use of intrinsic features from social networks enriches the proposal comparing to [21, 27, 44, 45].

In [23] is proposed BROAD-RS architecture that enables automation of Learning Objects recommendation process based on students interests and according to the class goals specified by the teacher. This model is based on pedagogical issues and students’ interests, such as: language, media, interactivity level, interactivity type, learning level, age range, among others. The proposed approach makes use of a multi-agent system that takes into account educational models, teacher’s didactical plan and the student context to recommend the most appropriate learning object. This paper goes forward BROAD-RS [23], as it uses explicit extraction for students profile and context information, and not only local resources repository. Besides that, BROAD-RSI is concerned about non-formal education.

The use of context and information extracted from social network to support recommendation items is not new. Fritzen et al. [21], for example, presented a proposal that aimed to model context from social network messages, and used this information to improve the relevance of Web document retrieval, wishing to support collaborative learning. Our proposal advances in relation to Fritzen’s, because it is possible to generate specific and individualized educational recommendations, consistent with interests and preferences of any user in social network, even if he/she is not member of a group with educational discussions.

The recommendation strategy discussed in this paper seeks to benefit both the user profile construction, using data generated spontaneously through interaction in Social Networks, as well as the representation and connection of educational resources through Linked Data.

3 BROAD-RSI system

BROAD-RSI (Recommender System Based on Social Interactions) was planned to fulfill the following requirements: (1) identify features of the user profile and context based on spontaneously generated information through interactions on social networks; (2) enrich user profile and context to assist non educational resources selection and recommendation; (3) offer educational recommendations to users not necessarily registered in any course or institution, but interested in being trained and non-formal education; (4) indication of people with similar educational interests, seeking to create a network of common interests.

In the scope of this work, we consider educational interests as all contents user wants information about, i.e., any subject to learn about, without being related to knowledge areas of formal education. To fulfill these goals, we chose Facebook social network as experimental study for the reasons mentioned above, including its significant use among students and people in general.

BROAD-RSI user information extraction and use in the recommendation workflow are showed in Fig. 1. The workflow begins with user accessing BROAD-RSI with Facebook account. Then he/she is asked to authorize the use of information profile. When user gives permission to employ his/her data, the information extraction step is executed, including profile information, friends list, liked pages, posts and groups. In the next step information are filtered, seeking for those that define educational interests, access schedule (day and time) preferences, languages, media, and also the profile data, such as name, email, age and sex. Once filtered the necessary information, identified interests undergo an enrichment step, which matches related interests, through some techniques of Information Extraction and Semantic Web representation.

Fig. 1

BROAD-RSI recommendation workflow

After interests enrichment step comes the semantic representations of these interests and user’s profile and context. Finally, profile and context are used to define search parameters, relevance of educational resources and also people with similar interests. The next steps include the discovery of recommender resources, their priority and finally send them through an application on the social network.

3.1 Infrastructure

BROAD-RSI was defined to process information extracted from a social network, set user profile and context features, and generate educational recommendations adherent to these features. The recommender infrastructure is divided into four main tiers (Fig. 2): Information Extraction; Profile Enrichment; Semantic Representation; and Recommender System.

Fig. 2

BROAD-RSI Infrastructure and Implementation technologies

This infrastructure aims to fulfill the following requirements:

• (1) specify a strategy to extract information from user profile and educational context, from Social Networks and Virtual Learning Environments;

• (2) identify user educational profile items from the extracted information;

• (3) enhance the extracted profile and context, seeking for implicit interest topics;

• (4) generate the semantic representation of the profile and context, and;

• (5) recommend appropriate educational resources considering the extracted information.

  BROAD-RSI prototype development required different technologies and Web solutions, as can be seen in Fig. 2. This figure presents the main used technologies in each architecture module. The proposed solution is based on Java programming language and Java computer platform.

3.1.1 Information extraction

The information extraction tier is responsible for identifying the available data, through users’ profile and their interactions in the social network, and then select those that are relevant to recommendation step and to match resources related to their educational interests.

Information extraction process starts with user access to Facebook account. As stated before, user must accept the required permissions to allow the information extraction.

Facebook provides, through its API, information about each user: public information from his/her profile (name, age, birth and home city, educational information, professional data, spoken languages); interests and preferences (books, music, movies, games, pages, so on.); and even the groups of which he/she participates.

Information definition process in BROAD-RSI system had two steps. At first, we searched for relevant information for educational recommender systems, i.e., which elements of user’s profile and context could be useful to generate relevant recommendations. We built a systematic mapping [13], that aimed to identify contextual elements in works related to sensitive-context on Educational Recommender Systems. The result was a quantitative analysis of context categories, displayed as list of information. In the second step, we applied a filter over the identified information in the first step, considering only those able to be extracted from Facebook. At the end of these two steps, contextual elements were defined (Table 1). Data were categorized according to [46] framework, as detailed in [13].

Table 1 BROAD-RSI contextual elements

User information on Facebook was accessed through the API for application developers, named GraphAPI version 2.0. This API is a platform for building applications. It allows the use of social links and profile information for the development of new applications. All developed applications must accept Facebook privacy terms. The GraphAPI uses RESTful protocol [47] and the answers are given in XML format. The exploitation of GraphAPI was made through the library named RestFB [48]. In addition to information related to users’ interests, we also captured those related to their access preferences, like days, times (morning, afternoon and evening) and dates. The system runs all the time in background mode, so the idea is to send educational recommendations during the most appropriate time, according to the user’s access profile. Device types used to access social network are extracted as well as the different types of media, obtained from the number of different types of shared and liked resources such as books, music, videos, images and games. These information helps to prioritize resources according to their data format. Therefore, a selected recommendation resource is always ranked in importance and priority.

In this tier, we also calculate two users’ contextual features: date and time that they have more interaction and preferred media. The first feature is calculated from the created_time and updated_time data of each post. In this calculation process are considered the last 50 user’s posts, and the day and time in which user has more posts are chosen as preferred day and time.

The second feature considers the number of pages related to books read (books.reads), watched movies (videos.watches), shared music (musics.listens), games (games.plays) and media type in shared posts. The most user-posted media types are selected as preferred media types. These two contextual features are regularly updated, adopting the interval of seven-week days.

Educational profile definition

After information be extracted it is necessary to filter them, looking for those of educational context. We considered: areas of interest defined in user’s Educational Profile and in “liked” pages whose category is related to educational subjects or interests, such as college pages, schools, research areas, educational sites, among others. It is also extracted the user school degree (High School, Undergraduate, Graduate, Postgraduate). We must emphasize that we considered educational subjects or interests all those that user wants to know or learn about, not restricted to formal education.

3.1.2 Profile enrichment

The profile enrichment tier is responsible for the identification of interests related to those explicitly informed in social networks profiles, through Information Extraction techniques and Semantic Web representation.

Pages, posts, groups and other information extracted from user profile, usually have a name and a text that describe them. In the first version, we considered only the name, title or the text about interest area, item or page. But there were not enough elements to identify user interest topics and generate relevant educational recommendations. The name or title generally brings little or oversimplified information, avoiding its use. On the other hand, the page text provides more comprehensive information that enables some associations. However, this information is distributed throughout the text without a predefined structure to allow its extraction.

Thus, it was necessary to search for Information Extraction techniques and Semantic Web representations, including ontologies. It aimed to reach relevant information and enrich profile and context items extracted from social network, obtaining a better understanding of interest issues and their association with correlated interests.

The enrichment approach using Information Extraction has the advantage of not depending on an expert in each knowledge area to create a domain ontology, so it can easily be applied in any knowledge domain. However, on the other hand, there are wide associations, bringing some results that, in some cases, may be less adherent to user interests. The solution adopted to reduce this problem was to allow users’ intervention to confirm or reject the identified interests.

The main Information Extraction tasks were: detection of entities, concepts, keywords and text language. The enrichment step using Semantic Web was done by specific domain ontologies for each expertise area. These two approaches have advantages and disadvantages, using both in a complementary way we can enrich user profile.

Using ontologies for the profile enrichment may provide a better precision in identifying related terms, since they are described by experts of each knowledge area. Another advantage is the predefined sequence of terms, as it can enrich the recommendations, defining a previous order to display the recommended resources. Both approaches can be used separately or in conjunction with entities, concepts and keywords extraction. If the pages do not show text or their text does not result in entities, concepts and keywords, an ontology is used to make queries, using the page title. Otherwise titles, entities, concepts and keywords, are used as parameters to search for correlated terms in the ontologies. Since they are complementary, when one is not sufficiently efficient for a given situation, the other approach is used to improve the results.

The enrichment approach using Information Extraction uses the AlchemyAPI (https://www.ibm.com/watson/services/natural-language-understanding/), it is a tool that provides a set of features to extract information from a text, such as tags extraction, language detection, entities extraction, text categorization, concepts extraction, keywords extraction, among others. Most of these features use statistical methods of natural language processing [49] and machine learning algorithms [50]. The choice of AlchemyAPI was supported by results present in [33] and [51], and because its integration with BROAD-RSI was easy considering it a web service. AlchemyAPI showed satisfactory results compared with other APIs, such as F-EXT, Ltasks and Boilerpipe. In addition, AlchemyAPI supports different languages, what was decisive in choosing it, since data extracted results from collaboration among people from different nationalities in social networks and it is possible to find pages in different languages.

The use of AlchemyAPI occurs after extraction of user favorite pages. At this point, it makes the descriptive text filtering of each page and then this text is submitted to the AlchemyAPI methods for language text detection and entity, concepts and keywords extraction.

For enrichment approach using ontologies the following tools were used Apache Jena, Sesame RDF (Resource Description Framework) database to create and manipulate ontologies and SPARQL language for queries.

Apache Jena (or only Jena) is a Java framework, open source, for building semantic web and linked data applications. The framework consists of different APIs that interact between them to process RDF data, a model that provides a way to express simple statements about resources, using named properties and values [52]. Sesame is a framework for storage and search large amount of metadata in RDF and RDF Schema, which supports storage, concurrency control, data export and queries in SPARQL. One can insert, delete, update and query RDF data. The RDF data of all ontologies used to support enrichment profile, user profile and recommendations are stored by Sesame. Queries in SPARQL language are performed to retrieve stored data and inferred interests. Ontologies used by BROAD-RSI were generated with the support of an application, also developed in the scope of this work. It extracts terms and their hierarchy from class contents of Federal University of Juiz de Fora courses, Brazil. Figures 3 and 4 show some examples.

Fig. 3

“Pharmacology” class Ontology

Fig. 4

“Software Engineering” class Ontology

We also used a computational taxonomy created from the ACM classification system [53]. It is based on a semantic vocabulary as a source of categories and concepts that reflects the Computer Science area state of the art. An example of SPARQL query made in this taxonomy can be seen in Fig. 5. In this query (left) the enrichment of “Data Mining” term is done, returning items related to it (right).

Fig. 5

Example of SPARQL in ACM taxonomy

Thus, if a user has interest in “Data Mining” area, either through actions that find it as a topic of his/her educational interest or giving a like in one page (https://www.facebook.com/groups/757135357709649/), then the interest will be extended to those terms shown in Fig. 5 (right), since they are all related to “Data Mining” area. Based on this enrichment the user will not only receive recommendations on the subject “Data Mining”, but also all the associated items.

3.1.3 Semantic representation

The semantic representation tier is responsible for creating the ontology of the user profile and interests by linking the extracted information to appropriate classes and properties. In addition, this component is responsible for storing the information in a semantic database.

After profile definition, identification and enrichment of user interests, their semantic representation takes place, including related properties which links users to their interests and to people they related.

The semantic representation becomes important for future inferences, associations and semantic search that can help both user profile and interest topics enrichment.

Educational and enriched interests extracted from Facebook also give rise to semantic-based educational interests, interrelated through semantic properties. For example, when an interest X is extracted from a user page, it is stored with a semantic link to user interest set. Discovery interests, after being enriched, are also stored and associated with interest X through semantic properties. This semantic representation is automatically generated because of BROAD-RSI usage.

In order to represent user profile and context vocabularies FOAF [54] and SIOC [55] ontologies were used with some extensions. FOAF (Friend of Friend) is an ontology, which uses RDF technology to semantically describe people, their interests, and their links on social networks. All extracted profile features from user’s profile on Facebook, the connection of these users with other users and topics of interest are represented using FOAF. The SIOC (Semantically-Interlinked Online Communities) ontology was used to represent all groups that user participates. It provides the main concepts and properties required to semantically describe online community information, such as forums, wikis, blogs and groups.

3.1.4 Recommendation approach

The recommendation tier is responsible for performing searches in different repositories (learning objects, linked data and videos), establishing the relationship between user profile and stored educational resources. This component also defines the recommendation priority based on the adherence of educational resources with user profile and context. In addition, this component is also responsible to recommend people with similar interests.

BROAD-RSI focus on the recommendation of educational resources stored in its own repository, and cataloged by a set of metadata, as detailed in [56]. However, the difficulty in locating educational resources in repositories, cataloging, and updating these data have significant impact on efficiency of Recommender Systems and of educational content personalization. Both depend on the quality and number of educational resources cataloged. In this way, we aim to search for alternatives to these repositories limitations, using other sources of educational resources. So, we present three recommendation approaches: based on learning objects repositories; based on linked data sets; and video recommendation based on YouTube.

Learning objects repositories

Educational resources recommendation in BROAD-RSI proposal considers the relations between user profile features and metadata of educational resources. Generally, educational resources are cataloged using metadata standards, such as IEEE LOM [57] and OBAA [58], allowing search and reuse of these resources.

Table 2 shows the adopted metadata, as well as established relations between them and extracted information.

Table 2 Relations between metadata and extracted information

First recommendation step is to seek for educational resources related to extracted user interests and the ones inferred from semantic enrichment step. This search returns a set of resources and then it processes the priority of the recommendations, i.e., the resources more adherent to user profile. The linkage is determined by comparing the values of metadata General.Language, Educational.InteractivityType and Technical.Format, Educational.Context, Educational.TypicalAgeRange, Technical.Requirement with spoken languages, favorite media, Education degree, age and used device, respectively.

For learning objects recommendation approach a relation between user profile and context features with metadata of educational resources is then established.

Considering the dynamics interaction in social networks, we adopted the prioritization of recent interests in recommendation process. We believe that as soon as users have expressed interest in a subject it is more relevant to send recommendations regarding this subject. Thus, most recently expressed are considered more relevant in sending recommendations.

Diversification is another issue addressed in recommendation approach. It is desirable that users receive recommendations on various subjects in which they have interests, not restricting the recommendations to one specific interest. In summary, the recommendation prioritization should follow two requirements: most recent interests will always be at first priority; interests that were awarded by (at least) one recommendation go to the end of priority list.

By performing these steps, we can ensure that user receives recommendations more related to his/her current interest, and also recommendations related to all interests.

Aiming to increase efficiency and accuracy in searching LO (Learning Object) metadata data, we used Lucene index search. It is an Open Source library for indexing and text querying. This library deals with two steps: (1) indexing, where original data is indexed generating a data structure related to keyword-based search; (2) index search, generating results ordered by similarity between text and user query. The similarity document is used in prioritizing the recommendation resources.

The index generation in Lucene was made as follows: for each record in the database with metadata values a distinct Document Lucene is generated, composed by five fields (identifier, title, description, keywords and a not indexed one that stores the address of accessed document).

After the filtering step, we need to define the adherence between the learning object and user profile and context, where it is checked whether the features of these objects are adhering to user profile and context items. Table 3 shows the percentages for definition of the rules, in order to define the levels of adherence. The TRUE (T) and FALSE (F) values are set from the comparison between the user features and the value of learning object metadata. For example, if learning object is written in English and user has English language skill we will assign T to the established relation, and so on for the other metadata.

Table 3 Adherence calculation of educational resources

After educational resources adherence calculation, relevance degree of the obtained document is applied through the similarity index filtering step using Lucene. Finally, the resources are sorted according to their adherence degree and document relevance in relation to user search. Table 4 shows an example of the educational resources range.

Table 4 Example of educational resources rank

The entire process is used in case the recommendations are automatically sent to user on the social network. If users want to see the recommendations available for a particular interest, step 1 is jumped, since users themselves can define the topics for which they wish to see the recommendations. Steps 2 and 3 continue to be adopted.

Overview of BROAD-RSI recommendation process is shown in Fig. 6.

Fig. 6

Overview of BROAD-RSI recommendation process

Linked data

The recommendation approach using linked data increases the possibility of finding educational content from different sources that are constantly being updated, avoiding the problem of new recommendations lack. Many repositories are beginning to provide their educational resources in Linked Data format, increasing the possibilities of educational resources location and recommendation in a wide area of interest.

In this work we use two content sources of Linked Data: DBpedia and Open University. DBpedia was chosen for three reasons: (1) large amount of information available; (2) possibility of recovering information using terms in different languages as search parameters; (3) diversity of themes for available information. Open University has the advantage of providing content with specific educational goals, what does not happen with DBpedia.

Among available information in both repositories, some of them may contribute to user learning process, such as a brief explanation on a particular subject, links to conferences, journals, pages that are associated with topics of interest, presentations, books, podcasts, videos, and lectures, among others.

The recommendation process starts by following the same steps of previous approach, choosing the topics of interest. After defining the recommendation subject, next step of the educational recommendation extraction through DBpedia is the search for pages related to selected topic of interest. Thus, a search using rdfs:labelproperty is executed. Resource labels of DBpedia are created from the titles of Wikipedia pages. Figure 7 is a query example for a search related to “Data Mining”.

Fig. 7

SPARQL for recovering resources in DBpedia

Once the query is finished, DBpedia service returns all resources which label contains the specified term. DBpedia page for “Data Mining” resource is shown in Fig. 8.

Fig. 8

DBpedia web page for “Data Mining”

After resource has been recovered, BROAD- RSI is responsible for performing other queries using specific properties which can bring relevant educational content to the topic of interest. Among available properties, the follow ones are used: dbpedia-owl:abstract , dbpedia-owl:wikiPageExternalLink , foaf:isPrimaryTopicOf , dbpedia-owl:academicDiscipline , dbpprop:discipline Of, dbpedia-owl:literaryGenre, dcterms:subject , dbpprop:hasPhotoCollection . These properties choice was made after a survey about which ones could return a relevant educational content. However, we cannot ensure that the list of properties has been fully considered.

Using these properties, we can retrieve and present information that can contribute to user’s learning process, as for example, a brief explanation of the topic, links to conferences, journals and other sites that are associated with the topic of interest. Figure 9 shows the screen displayed to user with links about “Data Mining”.

Fig. 9

Retrieved Links from DBPEDIA

A similar process is used for searching educational resources in the Open University dataset. However, in this case it is possible to use the rdfs:type property in order to group these resources and present them as groups, such as books, podcasts, documents in general, videos, courses, and so on. Figure 10 shows the SPARQL query executed in the Open University service to recover resources of Book type on “Project Management” theme.

Fig. 10

SPARQL performed in Open University dataset

The recovered resources can bring other related data that can be useful to the user. For example, a Book type resource can be part of a class or be linked with another resource type and thus increase user interest in learning content that go beyond the ones recovered by BROAD-RSI. In Fig. 11 we show some educational resources retrieved from Open University dataset.

Fig. 11

Educational Resources Recovered from Open University

In DBpedia and Open University, it is very difficult to prioritize the returned resources for each interest type, since they do not have a catalog with metadata that can be used to define those that are more relevant to the users. It makes difficult to automatically send the recommendation to the user and so we decided to make this functionality available only for recommendations requested by users.

In this approach, recommendations prioritization is a problem, since there is not a set of metadata that can be associated with the user profile and then define the adherence degree. Seeking to reduce this limitation, we used the social networks resources as follows: any educational resource is displayed with the options “like”, “recommend” or “share”, and also the number of people who had evaluated this resource. Thus, through BROAD-RSI, users have an indication of the number of people who have indicated or approved certain resource. The recommendations display can be seen in Figs. 12 and 13.

Fig. 12

Recommendation Links recovered from DBpedia (“Semantic Web” Theme)

Fig. 13

Recommendation Links recovered from DBpedia (“e-Learning” Theme)

Video recommendation

Seeking to diversify the sources of educational content for BROAD-RSI we also used YouTube. It offers many video resources and 300 h of video are uploaded every minute in more than 61 languages (fortunelords.com/youtube-statistics/). Among available videos, there are important production and provision of educational content like YouTube EDU. Therefore, it is possible to use this free channel of videos, encompassing a range of diverse topics to enrich the educational recommendations.

Through the API for developers, YouTube offers the possibility of many actions, such as searching and sending videos, playlist creation, registration, and so on. In this work the search considers user’s interests and spoken language. The search is made on the set of all videos available on YouTube. However, it is worth mentioning the possibility of making specific searches on a particular channel. It allows an institution, a teacher or a group of them to produce educational resources in video format, creating a specific channel of a particular subject. So BROAD-RSI can offer specific searches on that channel, thereby increasing the quality and accuracy of educational recommendations.

The video search is made with support of the YouTube video channel API. At first, BROAD-RSI administrator had to register the application on YouTube which returned a login that must be used to send requests. The search was only through keywords, indicating how many videos should be returned. During search configuration, one can specify the fields that will be considered: identifier, type, title, URL and video description.

The video relevance is defined by the API. In this approach, we do not consider other user profile features for prioritizing resources, since there are no metadata. The options “like”, “share” and “recommend” are also available for each of the recommended videos. The way the videos are displayed can be seen in Fig. 14.

Fig. 14

YouTube videos recommendation (“Data Mining” Theme)

In the case of videos, BROAD-RSI sends the recommendation considering the relevance of videos returned by the API, i.e., the most relevant video will be the first to be displayed.

People recommendation

People recommendation is made after identifying the users’ interest themes, being possible to extract networks of interest and indicate people with common interests. The construction of interest network provides a discussion environment of common topics, where members contributions may enrich all users’ knowledge.

Figure 15 shows the creation of an Interest Network in Computer Science theme, generated from information extracted from Facebook. The user represented by label (A) is one for which the recommendations will be sent. The nodes represented by label (B) are friends directly linked to A, i.e., people who have the same interest and are already part of the user’s friends group in Facebook. Nodes represented by label (C) are users who have the same interest and are not directly connected to A. It is possible to indicate friends who have common interests and also friends who can connect him/her to other people who have the same interests, but are not part of his/her friends group yet.

Fig. 15

Network of people with common interest

The list of recommended people is obtained by a semantic search in BROAD-RSI users’ database, considering similarity degree in relation to his/her interests. We consider all users’ interests for the similarity calculation, and users with substantial number of common interests are more similar among them. People recommendation is made by area of ​​interest, however, to calculate the similarity we consider all interests. For example, if user A asks for recommendation of people who have interest in Project Management area, BROAD-RSI will present other people who have this interest. For each recommended person, a numerical value that defines the similarity degree is shown. This calculation considers all common interests, as shown in eq. (1).

$$ SimilarityDegree=\frac{Total\ of\ common\ interests\ between\ A\ and\ B}{Total\ of\ interests\ of\ user\ A}, $$

(1)

where A is the user who requested the recommendation and B the recommended user.

If user A wants to see the network of interests around the theme “Software Engineering”, a semantic search will look for all users with interest in this topic. First the engine will find who are already his/her friends on Facebook. After, it will look for users not directly connected to user A, but with interest in Software Engineering. Finally, after querying for all these users, BROAD-RSI will identify the links between them. User A will be able to reach all the people who have interest in Software Engineering from their contacts on social network.

People recommendation development was made with support of a tool named Prefuse, which is an architecture for viewing graphs and trees developed in Java programming language. It has a set of features for data modeling, visualization and interaction. In this work, it was used in networks visualization as shown in Fig. 15.

3.1.5 Storage and delivery of recommendations

After performing all stages, the result is a set of educational resources (learning objects, links, videos, text) adherent and prioritized according to user profile and context. The recommendations are stored in a semantic database which will improve recommendations of BROAD Project.

The recommendation delivery can be made in two ways: (1) by user’s request and (2) automatically recommended in user’s social network.

In the first approach, user who wants to receive recommendations adherent to one specific interest, can access BROAD-RSI, choose the topic and then will receive a list of recommendations ordered by adherence to profile and context; and he/she will be able to evaluate by like, share or recommend each of them.

In the second approach, user will automatically receive recommendations in his/her profile on social network. The recommendation is selected based on the prioritization criteria, and the first is the more adherent to his/her profile, along with a link to evaluate it, as shown in Fig. 16. The options to share, like and recommend are those offered by social networks.

Fig. 16

User recommendation on social network

The recommendation is sent to user on Facebook, through API provided by this social network. After performing all the steps, two messages will be sent to user’s Facebook page: one with the summary of the topic of interest (extracted from DBpedia) and a learning object (extracted from the repository); and another message with the video (extracted from YouTube). Recommendations are always sent on day and time most accessed by the user.

After sending the recommendation, the resource is removed from the list of possible items to be recommended; thereby ensuring that user will not receive duplicate recommendations. User to see recommendations on any topics of interest, may access BROAD-RSI application main page and find recommendations listed by themes. All sent recommendations can be assessed by users through their links; or in BROAD-RSI application itself in the case of user’s request. The history of all recommendations is stored in a MySQL database, which also records user’s feedback for each recommended resource.

4 Evaluation process

The evaluation of BROAD-RSI infrastructure was done through three Proofs of Concept and one Case Study.

After developing the prototype, Proofs of Concept (PoC) were run with distinct goals, which will be detailed below. The goals of each proof of concept as well as the case study, were defined according to the Goal/Question/Metric approach (GQM) [59].

After finishing the Proofs of Concept execution, some adjustments were made to the prototype. It was also necessary to apply a questionnaire in order to clarify some issues that have arisen. Both the Proofs of Concept as the Case Study aimed to evaluate the proposal research question RQ1: If we spontaneously extract data generated from the user’s interaction in social networks, is it possible to identify features of user profile and context?; RQ2: If we use the extracted features of user’s profile and context from interaction on social networks, is it possible to perform relevant educational recommendations?

4.1 Proofs of concept

The decision to start the evaluation by a Proof of Concept is to demonstrate the technical feasibility of the proposed architecture, the concepts and technologies involved in the project. Although not presenting the formalism of an experimental study, the use of proof of concept also contributed to the preliminary evaluation of the questions about the evaluation instrument, the mitigation of specific technical difficulties and the refinement of the steps that were, subsequently applied in the case study.

The evaluation was performed with real users, identified by the pseudonyms: Researcher A and Researcher B, both students of the master program in Computer Science, Federal University of Juiz de Fora, Brazil. Both use Facebook daily, for more than 4 h. They use Facebook as one of their information sources on various subjects, including those for their educational career. During the proofs of concept, the users were asked to answer a few questions in order to evaluate different issues. The scale for data collection was the Likert scale [60]. The legends were fully agree, partially agree, neither agree nor disagree, partially disagree and fully disagree, associated with the numerical values 2, 1, 0, − 1, − 2, respectively.

4.1.1 First PoC: User profile and context features

The main goal of the first PoC is to extract user profile and context from Facebook, identify user’s educational interest trough his/her interactions and validate them.

Through the extracted profile features it was possible to obtain relevant information to the selection of educational resources. For example, Researcher A prefers textual educational resources, written in Portuguese or Spanish, with difficulty degree of post-graduate courses. It was also identified that Sunday is preferred day to access social network, around 11:00 am.

For Researcher B resources must be in audio format, which also have language and content adapted to post-graduate courses and the best day and time to send them is Tuesdays at 8:00 am. He/she has not indicated which are the favorite languages, however, by identifying the pages liked and shared we inferred this information.

In addition to profile features BROAD-RSI extracted users’ educational and major interests. The first step to discover user’s interests is to analyze all favorite pages and extract addressed areas. We used techniques of information extraction in order to categorize the pages. The result of the categorization of all Researcher A pages was the discovery of three major areas of interest (Fig. 17).

Fig. 17

Research A major interest areas

After major interests discovery, specific educational ones from each of the favorite pages are extracted, which will be the basis for the selection of topics of educational resources in future recommendations.

In addition to the researchers’ specific educational interests, BROAD-RSI was able to identify several other interests, filtered by pre-selected categories. Each page on Facebook has a category or set of categories. The selection of pages considered in the extraction of interests is based on their associated categories, in this case disregarding those not related to educational issues.

During Researcher A information extraction, there were detected 26 possible educational interests. He/she was then asked to evaluate such interest, answering, the following question: this topic of interest identified through your interactions on the social network Facebook is really one of your educational interests?

Combining the evaluations where Researcher A answered fully agree 29% and partly agree 46% for the extracted interests, we have a total of 75% of interest consistent with his/her actual educational interests. Researcher A indicated fully disagree 11% and partially 8%. He/she also declared neutral in relation to 6% of the extracted interests, as can be seen in Fig. 18.

Fig. 18

Extracted interests Results from Researcher A

In Researcher B profile there were identified 17 interests. He/she was also asked to answer the same question for each of the themes. The result of this evaluation is shown in Fig. 19. He/she totally agreed (35%) and partially agreed 29%, it has a total of 64% of topics consistent with his/her educational interests. Researcher B totally disagreed 6%, partially disagreed 12% and remained neutral in 18% of topics of interests.

Fig. 19

Extracted Interests Results from Researcher B

Regarding the extracted profile features, both users Researcher A and B fully agreed with all the extracted items.

This first proof of concept shows the feasibility of extracting information related to users’ interactions in Facebook, filtering this information so that profile features and educational interests are identified. Thus, there is evidence that the first research question (RQ1) can be confirmed, since the extraction of information, spontaneously generated from the interaction of users in social networks, allowed the identification of users’ profile and context features.

4.1.2 Second PoC: Enrichment of extracted educational interests

Whereas the extracted interests were enriched and new interests have been discovered, a second proof of concept was conducted specifically to analyze these discovered interests.

The main goal of the second PoC is to enrich the educational interest extracted from Facebook interaction, identify new interests and evaluate them by comparison with users’ real interests.

The interests extracted in the first proof of concept for Researcher A and Researcher B were enriched and some discovered interests were presented to them. During this proof of concept, it was also verified user acceptance for the two enrichment approaches: enrichment through information extraction techniques and through ontologies. In the first approach, enrichment is a result of using Information Extraction techniques for recognizing entities, concepts and keywords from the text description of the page of interest on Facebook, by using AlchemyAPI.

After the enrichment process, Researcher A and Researcher B were asked to rate some of the enriched interests, answering to the question: discovered interests or those extracted from your social network actually represent your real educational interests?

Users indicated totally 39% and partially agree 14% totalizing 53%. 23% neutral and partially disagreed 11% and fully disagreed 13%, as shown in Fig. 20.

Fig. 20

Researchers A and B Evaluation Results of the discovered interests

The comparison of the two approaches is shown in Fig. 21. The acceptance of the suggested interests through enrichment approach using ontologies was higher (60%) than the acceptance of the suggested interests through enrichment approach by information extraction techniques (50%).

Fig. 21

Comparison between enrichment techniques by ontology (outer circle) and Extraction Information (inner circle)

Regarding the general educational interests identified from the categorization of favorite pages, Researcher A indicated fully agree with 100% of interests, whereas Researcher B fully agreed with 50% interests and 50% partially disagreed with them.

By performing the second proof of concept it was possible to verify the viability of the discovery of interests that were not explicitly defined by users in their social networks. These interests can enrich and diversify the educational resources recommendations.

The results of this proof of concept reinforced the evidence obtained in the first PoC that the first research question can be confirmed. As well as being able to extract information from social networks, it was possible to enrich educational interests extracted from user social network through the discovery of implicit interests.

We observed that users were not familiar with some of the suggested themes, even those being directly linked to one of their topics of interest. In such cases, users indicated not to agree or disagree with the theme suggestion. It seems that although, at first, these issues have not been accepted, the recommendation of addressed educational resources can bring valuable information to users about a topic related to one of their interests.

There was a slightly higher acceptance regarding the inferred interests with the support of ontology than the interests found through information extraction techniques. We believe that it is true for other contexts scenarios, as the ontology is generally more accurate since it depends on expert knowledge, as information extractions automatic performed from the analysis of the texts, may suggest terms that do not really express a correlation with the user’s interest.

We also observed through the second PoC that enrichment by text extraction techniques was more efficient to interests described in English, as entity, concepts and their relationships and keywords are more complete for English terms. The use of ontologies can mitigate this situation through the creation of domain ontologies for each knowledge area in different languages.

The agreement of users with only 53% of discovered interests indicates that automating this part of the recommendation process may not be the best approach, since no validation by users in relation to discovered interests may generate recommendations that are not adherent to real user interests, and may not be achieved with the desired quality.

4.1.3 Third PoC: Suitable recommendations to user profile and context

The main goal of the third PoC is to use BROAD-RSI to recommend educational resources for Facebook users and evaluate their relevance. This Proof of Concept aims at assuring the adherence between educational resources features of and user’s preferences. Features of educational resources are recovered through the metadata. The relevance of the recommendation is its ability to meet a user need in searching information on a particular topic. That is, if the recommendation displayed will be relevance in user’s context somehow.

This proof of concept was performed with the same users described in PoC 1 and PoC 2. After identifying profile features, users’ educational interests and the discovery of new interests, BROAD-RSI derives some useful information to generate suitable recommendations for each user. To perform the third proof of concept each user chose some topics of interest among those extracted and enriched by BROAD-RSI and evaluated as fully or partially consistent with his/her actual educational interests.

The recommendations were made from three different sources: Group 1, learning objects stored in BROAD-RSI repository cataloged with the set of LOM metadata [57]; Group 2, recommendation links extracted from DBpedia [61] and Group 3 with extracted videos from YouTube. The quantitative results presented below show the groups separated for each kind of recommendation.

Researcher A and Researcher B were asked to rate the recommendations. To perform the evaluation, they answered the following questions: (1) Considering your interest in this subject, is the recommendation relevant to you?; (2) Would you share this recommendation in your social network?; (3) This recommendation is adherent to your profile?

Answers to the question 1 are shown in Table 5.

Table 5 Results from Researcher A answers to Question 1

Considering the three groups 80% of the recommendations were evaluated by Researcher A as fully agree (62%) or (18%) partially agree, considering the analyzed interest.

Researcher B answered the same question to all interests that were fully or partially consistent with his/her educational interests. Quantitative evaluation is shown in Table 6.

Table 6 Results from Researcher B answers to Question 1

The results pointed out that of the three groups, 83% of the recommendations were evaluated by Researcher B as fully agree (52%) and (31%) partially agree.

Comparing the three groups, Group 1 with the recommended items from BROAD-RSI repository, had the lowest acceptance percentage (fully agree + partially agree), 75%. Group 2 and Group 3 received acceptance (fully agree + partially agree) of over 80%.

The two participants were asked to answer also the following question: "Would you share this recommendation in your social network?". Researcher A answers can be seen in Table 7 and Researcher B are presented in Table 8.

Table 7 Researcher A answers to Question 2

Table 8 Researcher B answers to Question 2

Researcher A agrees to share 82% of the recommendations (fully agree + partially agree), while Researcher B agrees to share 82.75% of recommendations (fully agree + partially agree). Comparing the three groups, Group 1, showed the lowest acceptance percentage (fully agree + partially agree) of sharing the recommendations in social networks, 75% Group 2 and Group 3 received acceptance (fully agree + partially agree) over 80%. Researchers’ positive answers in relation to share educational resources recommendations through social networks showed the possibility of reaching people who are not users of BROAD-RSI.

The third question asked was: This recommendation is adherent to your profile? Researcher A answers are in Table 9, and the ones of Researcher B are shown in Table 10.

Table 9 Researcher A answers to Question 3

Table 10 Researcher B answers to Question 3

Considering the recommendations of the three groups Researcher A agrees (fully agree + partially agree) 84% of the recommendations were adherent to his/her profile. Researcher B agrees (fully agree + partially agree) that 82.75% of the recommendations are adhered to his/her profile.

In relation to compliance with recommendations to the user profile, Group 1 was again the one that had the lowest percentage of acceptance, 75%. Groups 2 and 3 reached 86% and 91% respectively.

Regarding the adherence of recommendations to the user profile, it is worth noting that in the case of Group 1 Researcher A pointed out that fully agrees with 61% of the recommendations, while for Group 2 and Group 3 the fully acceptance percentage with partially acceptance were 93% and 88%, respectively. We cannot say exactly if the language has been a decisive factor in this evaluation. However, Researcher A, who indicated Portuguese and Spanish as favorite languages, evaluated as partially adherent most links extracted from DBpedia and videos available in YouTube, which have English as their main language. While the educational resources extracted from BROAD-RSI repository, mainly in Portuguese, were evaluated as fully adherent.

The results of the third proof of concept, indicate that the second research question can be confirmed, as it was possible to generate relevant educational recommendations, using profile and context features extracted from user’s interactions on social networks, and these recommendations could be presented through an application within the social network.

It was also observed that the recommendations of educational resources coming from the Web, links in the dataset DBpedia, or videos available on a video channel, were well evaluated. It may indicate that it is interesting to diversify the sources of educational resources, not limited to repositories of learning objects which are difficult to be frequently maintained and updated.

In addition to enable the verification of the technical feasibility of the proposal and give the first results about its acceptance, the three proofs of concept helped the formulation of the questionnaire used as an evaluation tool in the case study. From the prototype used for the proofs of concept, it was possible to set some parameters for the evaluation step, solve technical problems and define some issues that could take part in a case study.

Therefore, the results of the three proofs of concept motivated the planning and execution of a case study to increase the observation and viability of the BROAD-RSI proposal.

4.2 Case study

According to [62], a case study should be used when the researcher has little control over events in real contexts and without full control over the variables. The advantage of a case study is that it is easier to plan and more realistic, but the disadvantage is that the results can hardly be generalized and are more difficult to interpret.

In the context of this research, the best evaluation technique is a case study, since it is an empirical research carried out in a real context where we want to investigate the users’ acceptance of the recommendation approach proposed by BROAD-RSI and get their feedback.

This case study is exploratory and descriptive. Exploratory activities were used to observe some aspects that were not yet clearly defined, while the descriptive activities were applied to phenomena with a defined research question, searching to confirm or deny it.

The descriptive research was used to observe the following phenomena:

Users’ acceptance in relation to the profile and context features identified through their social networks;

• Users’ acceptance in relation to educational recommendations made in accordance with their profile features and extracted context;

• Educational resources repositories, as local BROAD-RSI repository, linked data sets and YouTube videos, that generate more adherent recommendations to user profile and context.

The exploratory research was applied to observe the following issues:

• How users analyze the use of social networks to receive educational recommendations?

• What social networks are the most used?

• How long users interact on social networks?

• Which social networks users prefer to receive educational recommendations?

The goals of this case study were defined according to the Goal/Question/Metric (GQM) approach [59]. This case study has two main goals related to the extraction of information from users’ interactions in their social networks and recommendations offered from their profile and context features:

• i. Analyze the extraction of user profile and context for the purpose of evaluating the educational interests extracted in relation to the accuracy adherence from the user’s point of view in the context of Facebook users which are concerned about Computer Science and express this interest through social network interactions.

• ii. Analyze BROAD-RSI educational resources recommendations for the purpose of evaluating their adherence from user’s point of view in the context of Facebook users, which have interests in technology and express this interest through social network interactions.

The case study was conducted with 14 volunteer participants distributed among students of Master Degree in Computer Science from the Federal University of Juiz de Fora and Computer Science staff (system analysts and professors). The requirements for the selection of individuals were to have an account on the Facebook social network and express interests through their interactions in this social network.

The data were collected using a direct method [63], by means of semi structured interviews, since the focus is to perform qualitative and quantitative analyzes. Direct questions were used in the evaluation for each profile item, interest extracted and inferred, recommendation generated and also after using the BROAD-RSI. The open questions were planned to capture information about the case study not provided in multiple-choice questions. In addition to seek for users’ points of view about the system and its features, we also expected to pick up suggestions for improvement of the recommendation process. Answer options followed the Likert scale, as detailed ahead.

The interviews applied after using the BROAD-RSI were made through Google Docs, using the form creation tool.

To carry out the data collection from users perform, BROAD-RSI prototype was implemented with its own database developed using MYSQL. A form data was stored in a table, associated with the evaluation form and it automatically receives users’ responses.

4.2.1 Metrics

In [64] the evaluation methods for recommendations are summarized in three most commonly used key metrics for Recommender Systems: Precision, Recall and False Positive Rate. In this case study we used the Precision metric because it was not possible to calculate other values needed for the other metrics. This happens because we use resources from a set of linked data and a website videos catalog.

In addition, we want to evaluate the acceptance of the items that were recommended and their relevance to users. Thus, the Precision metric matches this case. This metric does not represent a percentage value [64]. Better Precision metric results must be closer to 1.

According to [65], Precision metric is also often used to evaluate information extraction systems, defined as the number of correctly extracted information from all the extracted information. Thus, the same metric is used to evaluate the extraction of profile features, the extracted interests and those discovered by the enrichment step.

The Recall metric could also be used for extraction of information systems, however, in the scope of this study it is not feasible because extracted information comes from social networks, where the total set of information is not well defined nor well quantified.

4.2.2 Data analysis

After collecting data from interviews and questionnaire evaluation, a survey captured the following items: (1) evaluation of profile features and interests extracted or discovered by the enrichment step and (2) evaluation of the recommendations.

Profile features, extracted and discovered interests

Participants of the case study were asked to evaluate the extracted profile items through their interactions on social network. The items evaluated were spoken languages, media preferences, educational degree, week day, and time of higher access. The analysis aimed to answer the question: If we extract data spontaneously generated from user’s interaction in social networks, is it possible to identify user’s profile and context features. Applying Precision metric to each of the evaluated items, the results are presented in Table 11.

Table 11 Evaluation of user’s extracted profile and context features

Participants of the case study also evaluated each extracted educational interest by answering the following question: Does this topic of interest identified through your interactions on the Facebook social network is one of your educational interests? Educational interest is any topic that one has interest in obtaining information or to learn something about it.

The 14 participants evaluated 255 extracted interests, repeating them only for different users. Table 12 shows the results using the Precision metric.

Table 12 Evaluation of extracted interests

Participants were also asked to rate each of the enriched interests through ontologies and information extraction techniques, answering the following question: Did discovered interests represent one of your real educational interest? Table 13 shows the results of the users’ evaluation, according to Precision metric. Second column shows the results for general interests, i.e., major areas of interest, extracted from the analysis of all pages. Third column shows the results for specific educational interests related to the extracted interests. 33 major interests were analyzed and 519 educational interests, 224 enriched from ontologies and 295 through information extraction techniques.

Table 13 Participants’ Evaluation for discovered interests

It was also analyzed the performance of the two approaches used in enrichment step: Ontologies and Information Extraction techniques. The results are shown in Table 14.

Table 14 Evaluation of discovered interests through different approaches

4.2.3 Recommendations

To evaluate the recommendation process, the participants were asked to evaluate the relevance of each recommendation, answering the question: considering that you are looking for educational resources, is this recommendation relevant to you? We also desired to find out if using the profile and context features extracted from user’s interaction on social networks, it is possible to recommend relevant educational resources.

The local BROAD-RSI educational resources repository had at the time of the evaluation, 60 educational resources related to Software Engineering, Computer Networks, Computer Graphics and Database.

The results of the recommendations evaluations, using Precision metric is shown in Table 15, separated by each of recommendation groups, as previously explained in the PoC3. Last column presents the total results including all the groups. There were analyzed 427 recommendations, 84 of Group 1, 104 of Group 2, and 239 of Group 3.

Table 15 Results of recommendation evaluation

Two other questions were asked in order to define the recommendations reach ability, and whether recommendations proposed by BROAD-RSI were consistent with user profile and context features (Table 16).

• Question 1: Would you share this recommendation in your social network?

• Question 2: Does this recommendation is adherent to your profile?

Table 16 Results for Question 1 and Question 2

4.2.4 Results analysis

Analyzing the results of the case study, the extraction of users’ profile features was well evaluated by participants, and access time was the item with lower acceptance, reaching 0.62 of acceptance (partially agree + fully agree). This result showed 0.23 of neutrality. The questionnaire pointed out that most users could not evaluate this item because they had not clear perception of their interacting time on Facebook.

Regarding extracted interests there was also good acceptance reaching the rate of 0.69 against 0.22 of rejection, as Precision metric results close to 1 indicates better recommendation system. It provides evidence that the selection of the pages categories was satisfactory to filter users’ educational interests.

The major interests discovered had acceptance index of 0.75 against 0.21 of rejection, while the educational interests had an approval rating of 0.63 against 0.15 of rejection, in this case the neutrality index was 0.22. Acceptances rate observed in the case study were higher than those observed in PoC2. However, it is important to consider the possibility of requesting confirmation from users regarding the interests, seeking to reduce the number of recommendations of non-interest subjects. We also observed a similar behavior in PoC2 in relation to neutral ratings, i.e., when the user was unaware of a particular topic suggested he/she remained neutral, even though this issue was directly related to a subject of interest.

Comparing the two enrichment approaches, there was a difference in the case study results regarding PoC2. In the case study, there was an acceptance of 0.66 of items obtained by information extraction techniques, while the enrichment through ontologies reached 0.59 acceptance rate. We observed during the case study that some users’ “like” pages that address broad issues, e.g. Information Systems, but had specific interests, such as Web Development. The enrichment process with ontology brought all topics related to Information Systems, and most of them were rejected. On the other hand, Information Extraction techniques brought only interests possible to be extracted from the text of the page Information Systems. Those terms are usually general, but users evaluated them as accepted. From the results, it is not possible to say which approach is more efficient, reinforcing the strategy of using both in a complementary way.

In general, the recommendations had 0.65 of approval and 0.25 of rejection, considering the relevance in case of user search by educational resources in a given subject. When separated by groups, the approval ratings for Group 1, Group 2 and Group 3 were 0.58, 0.73, and 0.65 respectively. We believe that the low local repository acceptance rate is due to the number of educational resources available for recommendation because there were few resources to fit most extracted and enriched interests. In relation to the adherence of recommendations to user profile, an index of 0.66 of acceptance and 0.25 of rejection was obtained.

Participants were asked about their usage habits on social networks and about 55% indicated 4 to 6 h a day browsing or interacting on social networks, whereas the others indicated to spend from 2 to 4 h. It reinforces the importance of using the users’ spending/interacting time in their social networks to provide educational recommendations that may contribute to their knowledge skill. In addition, some users pointed out that others social networks besides Facebook should be considered as Twitter, Instagram, LinkedIn and YouTube.

Some users pointed out that it would be interesting to receive recommendations in the Twitter social network by the frequency that they use it, LinkedIn because it is a social network focused on professional field, with many educational and professional interests available in their profiles. One of the participants suggestion is:

“I believe that the use of other social networks like LinkedIn and ResearchGate, along with Facebook, can improve the tool, as they are social networks focused on job, research and academic world...”

It is possible to expand our proposal to other social networks, being necessary the use of an API for each social network and also suit the system to display the available information.

The participants approved recommendations on their social networks, about 90% indicated fully agree when asked whether they find interesting to receive recommendations on Facebook. Participants also said that they want to continue using BROAD-RSI:

“BROAD-RSI recommended many related resources, increasing the range of recommendations, and the recommendations interested me.”

“The suggested contents were closely related to my topics of interest, I considered a very important proposal for Education. The use of Facebook was a great idea; it is a network that can be used for interests and friendships.”

The local repository limitation in relation to the number of available educational resources and to suite topics of interest was also noted and pointed out by users:

“The repository content limits a more consistent recommendation for some of the subjects. It was possible to see that the recommendation works, but when they come from the repository there was a lack of resources.”

“In the case of the repository, it recommended me many subjects that were from computer area, but they were not within the subject I was searching.”

“The results of the repository were very specific, when subjects were more general the recommendations were not good or were displayed no recommendations.”

Participants pointed out that the educational resources available on video and text are their favorite ones. All users have pointed out two or more types of media for studying. This fact, together with the low fully acceptance rate (0.25) may indicate that the extraction of preferred media through “likes” and shared media probably do not represent all the users’ preferences. So, it is necessary, for example, to develop a ranking of the most used media types and evaluate its accuracy. There is also the possibility of using the media type of each well evaluated recommendation to create user’s preferred media parameter.

5 Final remarks

Diversity is one of the main features of the educational domain, so the definition of a recommender system not as a specific solution, but one proposal without strong compatibility restriction, may achieve different educational scenarios with learning goals, pedagogical approaches and specific organizational needs.

BROAD-RSI infrastructure aims to identify users’ profile and context features, interests and preferences, by extracting information from social network and enrich them through different technologies. We seek to explore educational purposes during users’ social network interactions, as well as use their spent time in these environments to send relevant educational recommendations from different educational resources.

A scientific research project has seven features described by DRESCH et al. (2015, apud HAVNER et al., 2014): (i) create an artifact to (II) address a particular problem, (III) which use must be explained by an appropriate evaluation of its applicability and (IV) the results and contributions must be shared with industry and academy. In order to ensure its validity, (V) investigations must be carried out with rigor and (VI) the possible solutions analyzed and, finally, (VII) the results should be communicated.

Thus, this research meets the seven main criteria in proposing (I) the BROAD-RSI to (II) favor the development of an educational recommender system able to extract users’ profile and educational context, from the Facebook social network and recommend educational resources (III) to evaluate it through the development of a prototype, three proofs of concept and a case study and (IV) present the results to the academic community and the industry who validated it and disseminating the results. We followed (V) methodological rigor during its development and evaluation, (VI) analyzing existing solutions and basing the proposal on learning objects repositories, linked data and video repositories and (VII) ensuring results publication.

There is relevant information in social networks that can assist in the task of identifying users profile and context features. The proposal proved the viability of sending educational recommendations to users on their social networks, using all available resources in these virtual environments, including their feedback. Through this research we could observe that:

• The automatic recommendation, considering users’ profile and context characteristics, can assist access to relevant educational information that may contribute to their academic achievement;

• Users’ profile and context characteristics in the generation of educational recommendations may stimulate their interest in a specific subject and help them to understand its content;

• The discovery of implicit interests related to those explicitly expressed through user’s interactions on social networks, allows recommendations generation on related subjects, expanding the possibility to access information that may be relevant to learning process;

• Extracting users’ profile and context characteristics from a social network assure that their personal data are always updated, even considering their educational interests as we can use Extraction Techniques and Semantic Web technologies to deal with the extraction, enrichment and for recommendations processes;

• The use of linked data set and a video repository on recommendation process increase the recommendations possibilities, not being restricted to a local repository of learning objects;

• Sending educational recommendations within social networks allows the use of the interaction possibilities of these virtual environments, share their experiences regarding a recommended educational resource and use the long time spent in social network to receive educational recommendations;

• The recommendation of people with common interests can contribute to information exchange among BROAD-RSI users, thus building new networks regarding a specific topic of interest.

The recommendation system brings innovative aspects in extracting and considering information spontaneously generated on social networks, as Facebook, and generating individual recommendations, specifically in Education domain, consistent with users’ identified interests and preferences. Aiming to create an educational environment of recommendation using information extracted from social networks, this research can be extended on various aspects such as:

• 1. expand the range of information used in extraction process, including individual posts in social networks, group posts and shares or through other systems that can increase this extraction process;

• 2. build new ontologies of different knowledge areas to compose our semantic database and allow new searches;

• 3. increase the use of linked data for the enrichment of interests;

• 4. expand the proposal to other social networks;

• 5. use software agents to allow the automation and integration between BROAD-RSI and tools used to Information Extraction and recommendation display;

• 6. refine and improve the recommendation approach considering people with common interests; users evaluation of BROAD-RSI recommendations; evaluation of users’ interaction in the BROAD-RSI through their logs.

Considering the information extraction, it would be important to expand the set of information, either through the inclusion of individual posts in social networks, group posts, shares, or through the inclusion of other systems such as virtual learning environments, academic systems and MOOCS support systems. In the case of inclusion of other systems it would be very interesting to also evaluate the quality of the profile extracted in the different systems.

The extraction of information can also be extended through new ontologies, from different knowledge areas, and it is worth emphasizing the importance of extending the entire extraction process to different languages, since used techniques showed good results for English and Portuguese languages, but were not extended to other languages.

The proposal was designed to undergo few adaptations in case of expansions to other networks, but it is necessary to consider the constrains, such as permissions and ways of information extraction.

Finally, our recommendation approach can be refined and improved, considering people network with common interests, users’ evaluation of the recommender resources and users logs, for example. The prototype can be refined by seeking improvements in the adherence calculation algorithm, considering educational resource, user profile and context and retrieving process.

Our resource group is also working on group recommender systems, as people with common interests may contribute to exchange information. Our challenge is the effective group categorization and the extraction of users’ interests as a member of this group.