dohuyhoangvn93
2 months ago
Hi HN,
I'm the author of DeepSearch.
I built this while dealing with large SMB shares over WiFi, where traditional directory scanning tools were either too slow or caused unnecessary network load.
The main idea is a layered scanning strategy: - Minimize SMB round-trips - Use Rayon for controlled parallelism - Avoid spawning excessive IO-bound tasks - Keep memory usage predictable
This is not meant to be a fancy UI tool, but a fast and deterministic building block that can be embedded into other systems.
The project is open-source (Apache 2.0). I'm especially interested in feedback on: - The concurrency model - SMB traversal strategy - Any edge cases you've seen in similar systems
Thanks for taking a look.
holg
2 months ago
Hi thx for sharing, in general
about concurrency Rayon excels on CPU-heavy tasks, but this is more IO
for traversal avoid any extra round-trips, simple the metadata already given
edge-cases would be infinite loops on symlinks, the Windows 255 char limits and on WIFI packet loss due to too many reqs
dohuyhoangvn93
a month ago
Thank you for your feedback. In the initial CLI test, I tried using Tokio and found that performance didn't improve and the earliest result return speed was quite poor. I skipped the root cause analysis and kept the original rayon design.
I will try redoing the tests and taking more detailed measurements for evaluation.
Thank you again.