Scenario 2 - Search analytics
Problem statement#
As data scientist, I need to extract and visualize insights from a document-based dataset to support reporting.
Goals#
- Load the data
- Provide a search engine tool to query the data
- The search engine tool is accessed by the user through a UI
- The search engine performances should be such that the user experience is smooth
- The retrieved data should also include a relevant visualizations
Tools & Capabilities#
| Tool | Description | Key capability |
|---|---|---|
| ElasticSearch | Elasticsearch is the distributed, RESTful search and analytics engine at the heart of the Elastic Stack. | - Data load - Provide a search engine tool - Search engine performances - Retrieve relevant data despite typos in the query keyword |
| Kibana | Kibana is your window into the Elastic Stack. Specifically, it is a browser-based analytics and search dashboard for Elasticsearch. | - Search dashboard for the search engine tool - Relevant visualization for the data retrieved |
Use case guide#
This document is meant to guide the user through Scenario 2 - Search Analytics, by presenting a high level overview of the main steps. As discussed in the use case description, the goal is to provide a search engine tool to query the dataset, enriching the results with some relevant data visualizations.
- Step 1: Initialize the resources. Launch two instances - ElasticSearch and Kibana - from the Service Catalog section of the Portal, and verify that their status is ACTIVE in the My Services section to double-check that the deployed instance is ready to be used.
- Step 2: Load the data. Access the Kibana instance through the self-service portal, log in with provided credentials, upload the downloaded dataset file in the "Get started by adding integrations" section, review the file details, import it, define an index name, create an index pattern, and navigate to Discover to view the imported data.
- Step 3: Perform data queries and related visualizations. Utilize the Discover function of Kibana to execute queries on the indexed data, extracting relevant information and insights by applying filters and analyzing available fields. Additionally, create visualizations for the data by generating charts that depict patterns and relationships, such as coordinator countries and funding schemes, as well as the status of researches based on programs, contributing to a comprehensive understanding of the dataset's characteristics and trends.