When preforming search with multiple search categories (search agents) and before result are loaded, initiation of new search with single search category takes too long to show search result on UI

MIP
, , , ,
1

I am developing multiple search agent plugins (6 plugins) that search for various categories of filters.

I am facing an issue where the new search result takes a long time to show when the previous search result is taking too long to render on the screen (it is being post-processed by Milestone) and I initiate a new search.

I have performed the below steps for the above-mentioned issue:

Prerequisite: The search scope is selected.

Steps:

1) Click on “Search for…”, select “Category 1” and click on “New Search”.

  • It will fetch 50,000 search results and pass them to the UI via “FireSearchResultsReadyEvent” and the UI will display the results.

2) Click on “Search for …” select all SIX categories ( which I have developed) and click on “New Search”.

  • Each category will return 50,000 results (so a total of 3,00,000 results) and pass them to the UI via the “FireSearchResultsReadyEvent” method from each agent’s Search method.
  • On UI loader is displayed and no search result is shown for a long duration due to post-processing by Milestone (at least for 15-20 minutes).

3) After performing Step 2 again and waiting for just a few seconds (no result as post-processing is going on by Milestone), perform similar steps as per Step 1, Click on “Search for…” select “Category 1” ad click on “New Search”.

  • It will fetch 50,000 search results and pass them to the UI. However, the UI is still showing the loader and not displaying the search result for a long time (at least for 10-15 minutes). which is shown immediately as per Step 1.
  • I believe it is waiting for the post-processing process of Step -2 to be completed before rendering the search result of Step 3.

It would be great if you could help me resolve this issue. I need search results on the UI on an immediate basis for “Step 3” just as in “Step 1,” which I am getting.

Let me know if any further details are required.

2

Thank you for reporting this. I have made a bug report with Milestone Development. I will get back when there is news from their investigation.

3

Are you able to share code so that we can reproduce this in the Milestone test lab?

I will be creating a support case where you can share source code for your plugin with Milestone Support directly and not on a public forum.

4

Thank you for your reply. I might not be able to share the code.

However, the use case is pretty simple.

I have developed SIX search agent plugins that return a maximum of 50,000 results; each has custom search result data with additional attributes in the result.

  1. When I select all SIX search agent plug-ins under category selection, all plug-ins will return 50,000 records, the UI shows loading, and no result is displayed due to post-processing done by Milestone to combine the results.

  2. After completing step 1, post-processing of results is still in progress (no result on the UI, and the loader is spinning). I select the single search agent plug-in and create a new search. The plug-in will return 50,000 records, but the UI takes more time (more than 10 minutes) to show results.

So if I directly perform Step 2 and skip Step 1, the result is shown in the UI immediately.

Note:

  1. Step 1 will take more than 15 minutes to show result on UI.
  2. Under SearchResultData object, BeginTime = TriggerTime - 5 seconds; EndTime = TriggerTime - 3 seconds; and TriggerTime = actual event time.
5

We have reproduced the issue in our end, and have learned that the issue is highly dependent on how close the search results are to each other in the timeline as results in the same timeframe are grouped by an expensive post-processing logic.

We would like to know how dense the results are in terms of timeline placement of the Evidence Lock results. Are all the 50,000 results happening within a day, a week, a month or more?

6

Yes, all 50,000 results will happen within a day or two. Sometimes it may take 2-4 days also or bit longer. Depending on certain conditions and configuration, it may vary.

Please do let me know if you need any other information.

7

Milestone Development have developed improved functionality, and this is currently in a review and testing phase. The expectation is to release it with 2024R1. (It will unfortunately not be ready in time for the upcoming 2023R3 release.)

8

@Bo Ellegård Andersen (Milestone Systems)​ Is there any update on the fix?

9

It is in 2024R1. I am not sure there is a date on release yet, I think the expectation is to release in June but please keep an eye out on news from Milestone, you should see it there first.