Hello Team,
We are experiencing an issue retrieving all document indexes from HighQ via API. The objective is to retrieve all active HighQ site names along with their respective file‑level details (metadata only). An earlier implementation successfully retrieved the full dataset; however, since around July 2025, the same approach no longer returns complete results. Running the existing solution without modification retrieves only ~3,000 documents, while the business confirms there are approximately 30,000 documents stored in HighQ. After refactoring the implementation to use async functions, the number of retrieved indexes increased to 8,000+, but the results remain incomplete.
The integration currently relies on the following HighQ API endpoints:
- The first one is to get all the Sites(https://collaborate.ocorian.com/estera/api/1/sites?filterby=all). We do a query where we exclude all archived sites

- The second one is to locate the items in the site folder.( https://collaborate.ocorian.com/estera/api/1/folders/ + si.sitefolderID + "/items";)

- The third call is to access the appropriate folder by using the folder id we get from the first call (“https://collaborate.ocorian.com/estera/api/1/folders/" + fol.id.ToString()).

- The last call is to get all the files details from that folder and insert it in our local database. (https://collaborate.ocorian.com/estera/api/1/files?q=\folderid=" + fol.id.ToString() + "\"")

Key observations from testing:
- Not retrieving the actual count of site details on HighQ
- Using Workato as an orchestration tool results in rate limit error, likely due to the volume of API calls
Could you please advise what is the best way for us to get all the indexes from HighQ(Site info and their respective files info only ) and save them in our local database

