This document describes and reports the results of the experimentation executed on the prototypes and the tools (semantic navigator, updated version of the semantic search, and new conceptual search) during the 2nd and 3rd development iterations.
Testing has been performed accessing the tools via Web Interfaces while the tools were installed on Telefonica premises.
The main goals of this experiment were:
- to verify in which way the recommendations provided with the first experimentation cycle were addressed: It was possible to verify that most of the suggestions about usability have been addressed, but the repository did not contain proper information in order to allow a significant testing of functionalities.
- to experiment the updated versions of tools in order to evaluate if they provided the expected functionalities and to evaluate the usability of the tools as well: The semantic navigator tool and the conceptual search tool have been experimented.
The results related the experimentation of the semantic navigator tool confirm the expectations, and the final assessment is that the tool addresses its requirements related to easy navigation through the entities stored in the semantic repository even if it should be improved for navigation of instances using a semantic search that allows to match the user requests with the stored data in a way that takes into account the meaning of the request.
About the conceptual search tool, it is highlighted that the experimentation has been performed on an initial release of this tool delivered in advance in order to have preliminary recommendations to be addressed in the final release. In any case, the experimentation didn't arise any fatal flaw and the tool operates accordingly to the requirements, even if it gave the chance to provide some suggestions for improvements, mainly focused on improvement related to the way to present the results because it is not straightforward to understand the relevance of the documents in the provided list of results.
It is noticed that the samples of documents used by the tool for performing its search is not well related to the vocabulary used for identifying the concept associated to the text submitted for search. In fact, the documents provide information about FLOSS while the vocabulary is not domain specific. This effectively limits the validity of the assessment.