One of Domino's main appdev scalability barriers with medium/large NSFs occurs when a user has access to a very small subset of documents. For example, consider the scenario:
1. A view or folder has tens of thousands of docs in it
2. A user only has access to a small (e.g. 1-10) docs within that
Any lookups, searches, GetDocumentByKey, etc. against that view or folder will result in major page-scanning operations as Domino tries to find docs. This can cause painfully long delays in code execution as Domino searches the entire view (maxing out CPU) to end up with the small set of docs that the user can actually see. This cost is incurred each time the code is run.
It would be awesome if NSF/NIF stored a "Readers/Authors" index that Domino would automatically use first to assist in the subsequent NIF scan operations.
DGQF seems to already have some capability near this area in that it seems "smart" in searches involving AND conditions. So there might not be a lot of extra work needed here. Such an index could probably be leveraged in many areas of the product for a significant performance/scalability boost.
If there are admin concerns (e.g. disk space, etc.) then this could perhaps be an NSF toggle.
We have certainly investigated such an index. It's in our backlog
@Thomas - DQL only uses the indexes that are there, and there is no Readers/Authors index. John Curtis @ HCL should be able to explain more, but the point is DQL suffers from this same scenario/problem I believe.
Moving idea to product : Domino Designer
Domino Query Language should provide you with the ability to search large amounts of data quickly.
Upgrade the NSF to support graph database functions (https://en.wikipedia.org/wiki/Graph_database).- this could give index free lookups