Week 11

by Vedant


Week Summary

Namaste 🙏 ,
Welcome to the eleventh weekly blog of GSoC’20 for CDLI.

What did you do this week?

This week, I completed the SQL query for an advanced search index. After indexing the data, the main task was to handle multiple fields of Advanced Search forms and integrating it with ES query to fetch results.

After integrating form and forming query in the required format, an issue arose regarding maximum hits (=10000) fetched from ES in one go. This issue was not addressed in Simple Search and postponed to address in the advanced search. So the rescuer was Scroll API, to fetch all the hits (> 10000) on the cost of heap memory. After optimising ES query, advanced search query was finally in a place with the optimised approach.

What is coming up next?

In the upcoming week, pagination for a large set of results will be implemented along with a filter option for search results.

Did you get stuck anywhere?

At a time from ElasticSearch, 10000 results can be fetched and to fetch more results comes at the cost of heap memory. The current index for advanced search contains almost 20 fields which require more memory for ElasticSearch to execute the search query. Finally, it was optimized by fetching only required fields in search results.

Branch (worked on):

  1. phoenix/feature/search
  2. phoenix/feature/advanceSearch

Issues :

  1. Closed or worked related to:
  1. Opened:

Pull Request :

  1. Merged (or under review):
  1. Reviewed:

Daily Work Update

# Day Date A short description of the work done
1 Monday 2020/08/10 a. WIP: Advance Search (SQL Query for ES index).
2 Tuesday 2020/08/11 a. WIP: Advance Search (SQL Query for ES index).
3 Wednesday 2020/08/12 a. WIP: Advance Search (SQL Query for ES index).
4 Thursday 2020/08/13 a. WIP: Advance Search (Backend).
5 Friday 2020/08/14 a. WIP: Advance Search (ElasticSearch Query).
6 Saturday 2020/08/15 a. WIP: Advance Search (ElasticSearch Query).
7 Sunday 2020/08/16 a. WIP: Advance Search.