Proposal: File Indexing and Fast Search Tool for TrueNAS SCALE

Problem / Justification

As an administrator managing dozens of TrueNAS servers, I regularly deal with massive amounts of user backup files. Finding a specific file can become time-consuming and inefficient, especially on large storage systems with millions of files.

While TrueNAS offers powerful storage and management features, it currently lacks a fast and simple built-in file indexing and search mechanism for large-scale datasets.

To address this need, I developed a Python-based indexing tool that efficiently scans file systems, stores metadata in a lightweight database, and allows instant searches based on various criteria such as name, size, type, or modification date.

Impact

The tool dramatically reduces search times across large NAS environments, improving both user experience and administrative efficiency.
It has already been adopted internally by several system administrators, proving its reliability and scalability.

Integrating such a solution into TrueNAS SCALE would:

  • Provide administrators with instant file lookup capabilities.

  • Help troubleshoot user requests faster.

  • Minimize downtime and manual browsing.

  • Enhance the overall value and usability of TrueNAS in enterprise contexts.

User story

As a TrueNAS administrator, I want to be able to quickly search and locate files across large datasets using flexible criteria, so that I can respond to user requests faster, manage storage more efficiently, and avoid wasting time manually navigating complex directory structures.

Repository

GitHub: MDVAurelien/FileIndex

2 Likes

That’s already a feature in the making

FWIW, you can just use sVDEV. Or metadata-only L2ARC.

1 Like

Yes, it’s available for Mac OS, but for administrators using the CLI, it’s something that’s missing.

The new functionality will be a service that also has a socket REST interface for local administrator to use. Also the samba mdsearch command can be used locally as well to query the database and find file / content matches.

2 Likes

Ahhh wonderful! Excellent news! :star_struck:

That’s awesome. I have found foxtrot on the Mac to be well worth its license money for similar applications.

Is sist2 a good alternative until then?

Closing as the work is already in progress.