Suchergebnisse

Background Shared tasks and community challenges represent key instruments to promote research, collaboration and determine the state of the art of biomedical and chemical text mining technologies. Traditionally, such tasks relied on the comparison of automatically generated results against a so-called Gold Standard dataset of manually labelled textual data, regardless of efficiency and robustness of the underlying implementations. Due to the rapid growth of unstructured data collections, including patent databases and particularly the scientific literature, there is a pressing need to generate, assess and expose robust big data text mining solutions to semantically enrich documents in real time. To address this pressing need, a novel track called "Technical interoperability and performance of annotation servers" was launched under the umbrella of the BioCreative text mining evaluation effort. The aim of this track was to enable the continuous assessment of technical aspects of text annotation web servers, specifically of online biomedical named entity recognition systems of interest for medicinal chemistry applications. Results A total of 15 out of 26 registered teams successfully implemented online annotation servers. They returned predictions during a two-month period in predefined formats and were evaluated through the BeCalm evaluation platform, specifically developed for this track. The track encompassed three levels of evaluation, i.e. data format considerations, technical metrics and functional specifications. Participating annotation servers were implemented in seven different programming languages and covered 12 general entity types. The continuous evaluation of server responses accounted for testing periods of low activity and moderate to high activity, encompassing overall 4,092,502 requests from three different document provider settings. The median response time was below 3.74 s, with a median of 10 annotations/document. Most of the servers showed great reliability and stability, being able to process over 100,000 requests in a 5-day period. Conclusions The presented track was a novel experimental task that systematically evaluated the technical performance aspects of online entity recognition systems. It raised the interest of a significant number of participants. Future editions of the competition will address the ability to process documents in bulk as well as to annotate full-text documents. ; This project has received funding from the European Union's Horizon 2020 research and innovation programme under Grant Agreement No. 654021 (OpenMinTeD), and the Encomienda MINETAD-CNIO as part of the Plan for the Advancement of Language Technology for funding. This work was partially supported by the Consellería de Educación, Universidades e Formación Profesional (Xunta de Galicia), under the scope of the strategic funding of ED431C2018/55-GRC Competitive Reference Group, and the Portuguese Foundation for Science and Technology (FCT), under the scope of the strategic funding of UID/BIO/04469/2013 unit and COMPETE 2020 (POCI-01-0145-FEDER-006684). ; Peer Reviewed ; Postprint (published version)

Background: Student participation and the use of active methodologies in classroom learning are being increasingly emphasized. The use of intelligent systems can be of great help when designing and developing these types of activities. Recently, emerging disciplines such as 'educational data mining' and 'learning analytics and knowledge' have provided clear examples of the importance of the use of artificial intelligence techniques in education. Objective: The main objective of this study was to gather expert opinions regarding the benefits of using complementary methods that are supported by intelligent systems, specifically, by intelligent information access systems, when processing texts written in natural language and the benefits of using these methods as companion tools to the learning activities that are employed by biomedical and health sciences teachers. Methods: Eleven teachers of degree courses who belonged to the Faculties of Biomedical Sciences (BS) and Health Sciences (HS) of a Spanish university in Madrid were individually interviewed. These interviews were conducted using a mixed methods questionnaire that included 66 predefined close-ended and open-ended questions. In our study, three intelligent information access systems (i.e., BioAnnote, CLEiM and MedCMap) were successfully used to evaluate the teacher's perceptions regarding the utility of these systems and their different methods in learning activities. Results: All teachers reported using active learning methods in the classroom, most of which were computer programs that were used for initially designing and later executing learning activities. All teachers used case-based learning methods in the classroom, with a specific emphasis on case reports written in Spanish and/or English. In general, few or none of the teachers were familiar with the technical terms related to the technologies used for these activities such as "intelligent systems" or "concept/mental maps". However, they clearly realized the potential applicability of such approaches in both the preparation and the effective use of these activities in the classroom. Specifically, the themes highlighted by a greater number of teachers after analyzing the responses to the open-ended questions were the usefulness of BioAnnote system to provide reliable sources of medical information and the usefulness of the bilingual nature of CLEiM system for learning medical terminology in English. Conclusions: Three intelligent information access systems were successfully used to evaluate the teacher's perceptions regarding the utility of these systems in learning activities. The results of this study showed that integration of reliable sources of information, bilingualism and selective annotation of concepts were the most valued features by the teachers, who also considered the incorporation of these systems into learning activities to be potentially very useful. In addition, in the context of our experimental conditions, our work provides useful insights into the way to appropriately integrate this type of intelligent information access systems into learning activities, revealing key themes to consider when developing such approaches. ; Spanish Ministry of Science and Innovation, European Regional Development Fund (ERDF) (TIN-2013-47153-C03-0X, X=1, 2, 3) and the Regional Government of Madrid under the Research Network MA2VICMR (S2009/TIC-1542). ; 2.731 JCR (2018) Q2, 57/155 Computer Science, Information Systems, 28/98 Health Care Science & Services, 11/26 Medical Informatics ; 0.960 SJR (2018) Q1, 10/163 Health Informatics ; No data IDR 2018 ; UEM