Speeding up Getting all Files in Directory after a Certain Date

There was a typo below, the first time is using LINQ, the second one is using Directory.GetFiles (then getting the time on each file within a loop). Big Difference, right? The last one is 41 times slower. The difference was not really using LINQ, but using the DirectoryInfo.GetFiles instead of the combination of Directory.GetFiles and then using File.GetCreationTime inside the loop.

DirInfo_SpeedUp_Stats_3

I was using the third method, and was getting so bored and tired of waiting on it to run when dealing with a file with about 5000 files in it, out of which about 450 matched my file mask. I knew it shouldn’t take that long to enumerate through a directory. So I did some searching and found this question on Stackoverflow.

 

The code below will benchmark all three methods:
1) LINQ (which uses DirectoryInfo.GetFiles)
2) DirectoryInfo.GetFiles
3) Directory.GetFiles and File.GetCreationTime

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO; 

namespace DirGetFilesSpeedTest
{
    class Program
    {
        static void Main(string[] args)
        {

            DateTime somedate = DateTime.Now.Subtract(new TimeSpan(24, 0, 0));
            string folderName = @"\\MyServer\Biztalk_Messages\Archive\856_EDI";

            DateTime startDateTime = DateTime.Now; 
            Console.WriteLine("Start LINQ query at " + DateTime.Now);
            DirectoryInfo dirInfo = new DirectoryInfo(folderName); 
            int countMatch = 0;
            var filterFiles = from file in dirInfo.GetFiles()
                              where file.CreationTime > somedate
                              select file;
            DateTime stopDateTime = DateTime.Now; 
            TimeSpan tsElapsed  = stopDateTime.Subtract(startDateTime);
            foreach (var x in filterFiles)
            {
                countMatch++; 
            }

            Console.WriteLine("CountMatching Files=" + countMatch);
            Console.WriteLine("Stop LINQ query at " + stopDateTime + " Duration = " + tsElapsed.TotalSeconds);

            startDateTime = DateTime.Now;
            FileInfo[] fileInfoArray = dirInfo.GetFiles("*.*");
            countMatch = 0;
            foreach (FileInfo fileInfo in fileInfoArray)
            {
                DateTime fileDateTime = fileInfo.CreationTime; 
                TimeSpan ts1 = DateTime.Now.Subtract(fileDateTime);
                if (ts1.TotalSeconds < 24 * 60 * 60)
                {
                    countMatch++;
                }
            }

            Console.WriteLine("CountMatching Files=" + countMatch);
            Console.WriteLine("Stop DirInfo loop  " + stopDateTime + " Duration = " + tsElapsed.TotalSeconds);

            
            startDateTime = DateTime.Now;
            string[] filenameArray = Directory.GetFiles(folderName, "*.*");
            countMatch = 0; 
            foreach (string filename in filenameArray)
            {
                DateTime fileDateTime = File.GetCreationTime(filename);
                TimeSpan ts1 = DateTime.Now.Subtract(fileDateTime);
                if (ts1.TotalSeconds < 24 * 60 * 60)
                {
                    countMatch++; 
                }
            }

            Console.WriteLine("CountMatching Files=" + countMatch);
            stopDateTime = DateTime.Now;
            tsElapsed = stopDateTime.Subtract(startDateTime);
            Console.WriteLine("Stop LINQ query at " + stopDateTime + " Duration = " + tsElapsed.TotalSeconds);
             

            Console.WriteLine("\n\nPress enter to end:"); 
            Console.ReadLine(); 

        }
    }
}

Uncategorized  

Leave a Reply