PowerShell Problem Solver: How to Pull Data from Log Files using PowerShell

In the world of Windows, a great deal of information is stored away in log files, which can easily be used with PowerShell. If the log files are structured or predictable, it doesn’t take much to create a PowerShell tool for extracting useful information. Many times you can get what you need with a simple one-line command. If this is a process that’s worth repeating or something you have to entrust to someone else, then you’ll want create a re-usable tool. In this article, I’ll show you how to easily pull data from log files using PowerShell, with some additional help from the Get-Content and Import-CSV cmdlets.
Before we even begin, please allow me to stress that even though we will be extracting data from text files, don’t think in terms of textual output. Always be thinking about objects in the pipeline. If you’re thinking in this way, then it’s much easier to get any specific text values that you need.

Let’s use the Windows Update log as an example, since it is something everyone should have. First, what does the data look like?

get-content C:windowsWindowsUpdate.log -TotalCount 5

 
I grabbed the first few lines of the file. Although there is no header, the file appears to have some structure in what is a probably a tab delimited format. Clearly there are some date and time values, as well as message strings. Rule number one is to know your data. In this particular case, Microsoft has provided some documentation on the file format.
Although there’s no header, the data is structured, and I should be able to use Import-CSV to import it. The following lines lets me define and provide my own header.

$header = "Date","Time","PID","TID","Component","Message"
$log = import-csv -Delimiter `t -Header $header -Path C:windowsWindowsUpdate.log

Look at what I get in $log:

I now have objects that I can use like anything else in PowerShell.

For example, the following line of PowerShell let’s us find all fatal problems.

$log | where {$_.message -match "fatal"} | out-gridview -title "Fatal Errors"

There’s potential pitfall, however, especially if you plan on filtering or sorting. Let’s look again at the resulting object in $log:

With Import-CSV, everything becomes a string. This means you have to take an extra step or two to get the data into a format that’s easier to use. Here’s what I can do with the Windows update log.

$header = "Date","Time","PID","TID","Component","Message"
$log = import-csv -Delimiter `t -Header $header -Path C:windowsWindowsUpdate.log |
Select-Object @{Name="DateTime";Expression={
"$($_.date) $($_.time.Substring(0,8))" -as [datetime]}},
@{Name="PID";Expression={$_.PID -as [int]}},
@{Name="TID";Expression={$_.TID -as [int]}},Component,Message

The import is very similar to what I did earlier, except that I’m creating another property called DateTime that will be constructed from the incoming Date and Time properties.

In this scenario, I found I needed to strip off the milliseconds from the time stamp in order for PowerShell to format the result as a DateTime object. But now I have something much more useful.

Now I can more easily find the information I need.

$log | where { $_.Datetime -gt "1/5/2015" -AND $_.component -eq "Agent"} | Out-gridview

This works fine locally, but what if I want to able to search update logs on remote computers? This is where scripting comes in handy. Using PowerShell remoting I can easily run the same command in a remote session.

#requires -version 3.0
Function Get-WindowsUpdateLog {
[cmdletbinding()]
Param(
[Parameter(Position=0,ValueFromPipeline)]
[ValidateNotNullorEmpty()]
[string[]]$Computername = $env:COMPUTERNAME
)
Begin {
    Write-Verbose "Starting $($MyInvocation.Mycommand)"
    $header = "Date","Time","PID","TID","Component","Message"
} #begin
Process {
    Write-Verbose "Processing Windows Update Log on $($($computername.toUpper()) -join ",")"
    #define a scriptblock to run remotely
    $sb = {
    Import-Csv -Delimiter `t -Header $using:header -Path C:windowsWindowsUpdate.log |
    Select-Object @{Name="DateTime";Expression={
    "$($_.date) $($_.time.Substring(0,8))" -as [datetime]}},
    @{Name="PID";Expression={$_.PID -as [int]}},
    @{Name="TID";Expression={$_.TID -as [int]}},Component,Message
    }
    Try {
       Invoke-Command -ScriptBlock $sb -ComputerName $Computername -errorAction Stop |
       Select * -ExcludeProperty RunspaceID
    }
    Catch {
        Throw $_
    }
} #process
End {
    Write-Verbose "Ending $($MyInvocation.Mycommand)"
} #end
} #end function

All I’ve really done is take the code that I know already works and wrap it in a function that runs it remotely using Invoke-Command. The remote execution automatically incorporates a computername property.

$data = Get-WindowsUpdateLog -Computername chi-dc01,chi-dc02,chi-dc04 -verbose

Now I can slice and dice update data across multiple sources.

$data.where({$_.DateTime -ge "12/1/2014" -AND $_.Message -match "Fatal"})| Sort DateTime,PSComputername |
Select PSComputername,DateTime,Component,Message |
format-table –AutoSize

You should be able to use these techniques for any structured log file, with or without a header. The data is there, waiting for you to come and get it.