You will likely get a memory error (System.OutOfMemoryException) if you use Get-Content to try to load a 550MB file into a string variable. We can using a System.IO.StreamRead in PowerShell to stream the data instead of doing one big massive load into memory. The script below allows you to search a huge file for a string. For example, in this case I had an OpenSSH SFTP log with 4 months of data in it. I wanted to find the date/time when a user last attempted to connect.
<pre>
# Example from log file when a userid is matched
#10464 2020-03-31 13:47:30.273 debug1: user Walmart matched 'User Walmart ' at line 134
cls
Write-Host "Start"
#$logFile = "c:\ProgramData\ssh\logs\sshd_log_Split_19_2020_03_30_to_03_31.log"
#$logFile = "c:\ProgramData\ssh\logs\sshd_log_Split_20_2020_03_31_to_04_01.log"
$logFile = "c:\ProgramData\ssh\logs\sshd_log_Split_21_2020_04_01B.log"
#$logFile = "c:\ProgramData\ssh\logs\sshd_log_Split_22_2020_04_01_to_04_02.log"
$ErrorActionPreference = "Stop"
$reader = new-object System.IO.StreamReader($logFile)
$count = 1
$lineNumber = 1
while(($line = $reader.ReadLine()) -ne $null)
{
++$lineNumber
if ($line.ToLower().Indexof("user walmart matched") -ge 0)
{
++$count
Write-Host $count $lineNumber $line
}
}
$reader.Close()
Write-Host "End"
</pre>