If you are an IT pro, then you are most likely the IT pro that’s on call for your family, friends and neighbors. You get a call that a neighbor’s computer is running slow or experiencing odd behavior. Virus? Malware? Rootkit? Application issues? If you are also like me, then you tend to rely on a collection of free and incredibly useful tools like Trend Micro’s HouseCall, Hijack This or CCleaner. Perhaps you might even need a copy of the latest tools from the Sysinternals site. In the past I’ve grabbed a spare USB key, plugged it in and started downloading files. But this is a time consuming and boring process, which makes it a prime candidate for automation. And in my case that means PowerShell.
Using Invoke-WebRequest
PowerShell 3.0 brought us a new command, Invoke-WebRequest. This cmdlet eliminated the need to use the .NET Framework in scripts. We no longer needed to figure out how to use the Webclient class. Cmdlets are almost always easier to use. If you look at the help for Invoke-WebRequest, then you’ll see how easy it is. All you really need to specify is the URI to the web resource. So for my task all I need is a direct download link to the tool I want to grab.
Invoke-webrequest –uri http://go.trendmicro.com/housecall8/HousecallLauncher64.exe
However, in this situation, I don’t want to write the result to the PowerShell pipeline, I want to save it to a file. Invoke-Webrequest has a parameter for that.
Invoke-webrequest –uri http://go.trendmicro.com/housecall8/HousecallLauncher64.exe -outfile d:HouseCallx64.exe -DisableKeepAlive -UseBasicParsing
I am using a few other parameters since I’m not doing anything else with the connection once I’ve downloaded the file. This should also make this command safer to run in the PowerShell ISE on 3.0 systems. In v3 there was a nasty memory leak when using Invoke-Webrequest in the PowerShell ISE. That has been fixed in v4. So within a few seconds I have the setup file downloaded to drive D:. That is the central part of my download script.
#requires -version 3.0
#create a USB tool drive
<#
.Synopsis
Download tools from the Internet.
.Description
This command will download a set of troubleshooting and diagnostic tools from the Internet. The path will typically be a USB thumbdrive. If you use the -Sysinternals parameter, then all of the SysInternals utilities will also be downloaded to a subfolder called Sysinternals under the given path.
.Example
PS C:> c:scriptsget-mytools.ps1 -path G: -sysinternals
Download all tools from the web and the Sysinternals utilities. Save to drive G:.
.Notes
Last Updated: September 30, 2014
Version : 1.0
****************************************************************
* DO NOT USE IN A PRODUCTION ENVIRONMENT UNTIL YOU HAVE TESTED *
* THOROUGHLY IN A LAB ENVIRONMENT. USE AT YOUR OWN RISK. IF *
* YOU DO NOT UNDERSTAND WHAT THIS SCRIPT DOES OR HOW IT WORKS, *
* DO NOT USE IT OUTSIDE OF A SECURE, TEST SETTING. *
****************************************************************
.Link
Invoke-WebRequest
#>
[cmdletbinding(SupportsShouldProcess=$True)]
Param(
[Parameter(Position=0,Mandatory=$True,HelpMessage="Enter the download path")]
[ValidateScript({Test-Path $_})]
[string]$Path,
[switch]$Sysinternals
)
Write-verbose "Starting $($myinvocation.MyCommand)"
#hashtable of parameters to splat to Write-Progress
$progParam = @{
Activity = "$($myinvocation.MyCommand)"
Status = $Null
CurrentOperation = $Null
PercentComplete = 0
}
#region data
<#
csv data of downloads
The product should be a name or description of the tool.
The URI is a direct download link. The link must end in the executable file name (or zip or msi).
The file will be downloaded and saved locally using the last part of the URI.
#>
$csv = @"
product,uri
HouseCallx64,http://go.trendmicro.com/housecall8/HousecallLauncher64.exe
HouseCallx32,http://go.trendmicro.com/housecall8/HousecallLauncher.exe
"RootKit Buster x32",http://files.trendmicro.com/products/rootkitbuster/RootkitBusterV5.0-1180.exe
"Rootkit Buster x64",http://files.trendmicro.com/products/rootkitbuster/RootkitBusterV5.0-1180x64.exe
RUBotted,http://files.trendmicro.com/products/rubotted/RUBottedSetup.exe
"Hijack This",http://go.trendmicro.com/free-tools/hijackthis/HiJackThis.exe
WireSharkx64,http://wiresharkdownloads.riverbed.com/wireshark/win64/Wireshark-win64-1.12.1.exe
WireSharkx32,http://wiresharkdownloads.riverbed.com/wireshark/win32/Wireshark-win32-1.12.1.exe
"WireShark Portable",http://wiresharkdownloads.riverbed.com/wireshark/win32/WiresharkPortable-1.12.1.paf.exe
SpyBot,http://spybotupdates.com/files/spybot-2.4.exe
CCleaner,http://download.piriform.com/ccsetup418.exe
"Malware Bytes",http://data-cdn.mbamupdates.com/v2/mbam/consumer/data/mbam-setup-2.0.2.1012.exe
"Emisoft Emergency Kit",http://download11.emsisoft.com/EmsisoftEmergencyKit.zip
"Avast! Free AV",http://files.avast.com/iavs5x/avast_free_antivirus_setup.exe
"McAfee Stinger x32",http://downloadcenter.mcafee.com/products/mcafee-avert/Stinger/stinger32.exe
"McAfee Stinger x64",http://downloadcenter.mcafee.com/products/mcafee-avert/Stinger/stinger64.exe
"Microsoft Fixit Portable",http://download.microsoft.com/download/E/2/3/E237A32D-E0A9-4863-B864-9E820C1C6F9A/MicrosoftFixit-portable.exe
"Cain and Abel",http://www.oxid.it/downloads/ca_setup.exe
"@
#convert CSV data into objects
$download = $csv | ConvertFrom-Csv
#endregion
#region private function to download files
Function _download {
[cmdletbinding(SupportsShouldProcess=$True)]
Param(
[string]$Uri,
[string]$Path
)
$out = Join-path -path $path -child (split-path $uri -Leaf)
Write-Verbose "Downloading $uri to $out"
#hash table of parameters to splat to Invoke-Webrequest
$paramHash = @{
UseBasicParsing = $True
Uri = $uri
OutFile = $out
DisableKeepAlive = $True
ErrorAction = "Stop"
}
if ($PSCmdlet.ShouldProcess($uri)) {
Try {
Invoke-WebRequest @paramHash
get-item -Path $out
}
Catch {
Write-Warning "Failed to download $uri. $($_.exception.message)"
}
} #should process
} #end download function
#endregion
#region process CSV data
$i=0
foreach ($item in $download) {
$i++
$percent = ($i/$download.count) * 100
Write-Verbose "Downloading $($item.product)"
$progParam.status = $item.Product
$progParam.currentOperation = $item.uri
$progParam.PercentComplete = $percent
Write-Progress @progParam
_download -Uri $item.uri -Path $path
} #foreach item
#endregion
#region SysInternals
if ($Sysinternals) {
#test if subfolder exists and create it if missing
$sub = Join-Path -Path $path -ChildPath Sysinternals
if (-Not (Test-Path -Path $sub)) {
mkdir $sub | Out-Null
}
#get the page
$sysint = invoke-webrequest "http://live.sysinternals.com/Tools" -DisableKeepAlive -UseBasicParsing
#get the links
$links = $sysint.links | Select -Skip 1
#reset counter
$i=0
foreach ($item in $links) {
#download files to subfolder
$uri = "http://live.sysinternals.com$($item.href)"
$i++
$percent = ($i/$links.count) * 100
Write-Verbose "Downloading $uri"
$progParam.status ="SysInternals"
$progParam.currentOperation = $item.innerText
$progParam.PercentComplete = $percent
Write-Progress @progParam
_download -Uri $uri -Path $sub
} #foreach
} #if SysInternals
#endregion
Write-verbose "Finished $($myinvocation.MyCommand)"
Within the script, there’s a string of CSV data. The data contains a description and direct link for all the tools I want to download. You can add or delete these as you see fit. Just make sure the download link ends in a file name. The download function will parse out the last part of the URI and use it to create the local file name.
$out = Join-path -path $path -child (split-path $uri -Leaf)
All you need to do is specify the path, which will usually be a USB thumb drive.
The script has an optional parameter for downloading utilities from the Live.SysInernals.com website. If you opt for this, then the script will create a subfolder for SysInternals tools. That’s the way I like it. To download the tools I first use Invoke-WebRequest to get the listing page.
$sysint = invoke-webrequest "http://live.sysinternals.com/Tools" -DisableKeepAlive -UseBasicParsing
Within this object is a property called Links, which will have links to each tool.
$links = $sysint.links | Select -Skip 1
The first link is to the parent directory, which I don’t want which is why I’m skipping 1. Then for each link I can build the URI from the HREF property.
$uri = http://live.sysinternals.com$($item.href)
The only other thing I’ve done that you might not understand is that I’ve created a function with a non-standard name. I always try to avoid repeating commands or blocks of code. I created the _download function with the intent that it will never be exposed outside of the script. And this is a script which means to run it you need to specify the full path.
PS C:> c:scriptsget-mytools.ps1 -path G: -sysinternals
As I mentioned, I included the CSV data within the script which makes it very portable. But you might want to keep the download data separate from the script. In that case you’ll need a CSV file like this:
product,uri HouseCallx64,http://go.trendmicro.com/housecall8/HousecallLauncher64.exe HouseCallx32,http://go.trendmicro.com/housecall8/HousecallLauncher.exe "RootKit Buster x32",http://files.trendmicro.com/products/rootkitbuster/RootkitBusterV5.0-1180.exe "Rootkit Buster x64",http://files.trendmicro.com/products/rootkitbuster/RootkitBusterV5.0-1180x64.exe RUBotted,http://files.trendmicro.com/products/rubotted/RUBottedSetup.exe "Hijack This",http://go.trendmicro.com/free-tools/hijackthis/HiJackThis.exe WireSharkx64,http://wiresharkdownloads.riverbed.com/wireshark/win64/Wireshark-win64-1.12.1.exe WireSharkx32,http://wiresharkdownloads.riverbed.com/wireshark/win32/Wireshark-win32-1.12.1.exe "WireShark Portable",http://wiresharkdownloads.riverbed.com/wireshark/win32/WiresharkPortable-1.12.1.paf.exe SpyBot,http://spybotupdates.com/files/spybot-2.4.exe CCleaner,http://download.piriform.com/ccsetup418.exe "Malware Bytes",http://data-cdn.mbamupdates.com/v2/mbam/consumer/data/mbam-setup-2.0.2.1012.exe "Emisoft Emergency Kit",http://download11.emsisoft.com/EmsisoftEmergencyKit.zip "Avast! Free AV",http://files.avast.com/iavs5x/avast_free_antivirus_setup.exe "McAfee Stinger x32",http://downloadcenter.mcafee.com/products/mcafee-avert/Stinger/stinger32.exe "McAfee Stinger x64",http://downloadcenter.mcafee.com/products/mcafee-avert/Stinger/stinger64.exe "Microsoft Fixit Portable",http://download.microsoft.com/download/E/2/3/E237A32D-E0A9-4863-B864-9E820C1C6F9A/MicrosoftFixit-portable.exe "Cain and Abel",http://www.oxid.it/downloads/ca_setup.exe
And this version of the script.
#requires -version 3.0
#create a USB tool drive
Function Get-MyTool2 {
<#
.Synopsis
Download tools from the Internet.
.Description
This command will download a set of troubleshooting and diagnostic tools from the Internet. The path will typically be a USB thumbdrive. If you use the -Sysinternals parameter, then all of the SysInternals utilities will also be downloaded to a subfolder called Sysinternals under the given path.
.Example
PS C:> Import-csv c:scriptstools.csv | get-mytool2 -path G: -sysinternals
Import a CSV of tool data and pipe to this command. This will download all tools from the web and the Sysinternals utilities. Save to drive G:.
.Notes
Last Updated: September 30, 2014
Version : 1.0
****************************************************************
* DO NOT USE IN A PRODUCTION ENVIRONMENT UNTIL YOU HAVE TESTED *
* THOROUGHLY IN A LAB ENVIRONMENT. USE AT YOUR OWN RISK. IF *
* YOU DO NOT UNDERSTAND WHAT THIS SCRIPT DOES OR HOW IT WORKS, *
* DO NOT USE IT OUTSIDE OF A SECURE, TEST SETTING. *
****************************************************************
.Link
Invoke-WebRequest
#>
[cmdletbinding(SupportsShouldProcess=$True)]
Param(
[Parameter(Position=0,Mandatory=$True,HelpMessage="Enter the download path")]
[ValidateScript({Test-Path $_})]
[string]$Path,
[Parameter(Mandatory=$True,HelpMessage="Enter the tool's direct download URI",
ValueFromPipelineByPropertyName=$True)]
[ValidateNotNullorEmpty()]
[string]$URI,
[Parameter(Mandatory=$True,HelpMessage="Enter the name or tool description",
ValueFromPipelineByPropertyName=$True)]
[ValidateNotNullorEmpty()]
[string]$Product,
[switch]$Sysinternals
)
Begin {
Write-Verbose "Starting $($myinvocation.MyCommand)"
#hashtable of parameters to splat to Write-Progress
$progParam = @{
Activity = "$($myinvocation.MyCommand)"
Status = $Null
CurrentOperation = $Null
PercentComplete = 0
}
Function _download {
[cmdletbinding(SupportsShouldProcess=$True)]
Param(
[string]$Uri,
[string]$Path
)
$out = Join-path -path $path -child (split-path $uri -Leaf)
Write-Verbose "Downloading $uri to $out"
#hash table of parameters to splat to Invoke-Webrequest
$paramHash = @{
UseBasicParsing = $True
Uri = $uri
OutFile = $out
DisableKeepAlive = $True
ErrorAction = "Stop"
}
if ($PSCmdlet.ShouldProcess($uri)) {
Try {
Invoke-WebRequest @paramHash
get-item -Path $out
}
Catch {
Write-Warning "Failed to download $uri. $($_.exception.message)"
}
} #should process
} #end download function
} #begin
Process {
Write-Verbose "Downloading $product"
$progParam.status = $Product
$progParam.currentOperation = $uri
Write-Progress @progParam
_download -Uri $uri -Path $path
} #process
End {
if ($Sysinternals) {
#test if subfolder exists and create it if missing
$sub = Join-Path -Path $path -ChildPath Sysinternals
if (-Not (Test-Path -Path $sub)) {
mkdir $sub | Out-Null
}
#get the page
$sysint = Invoke-WebRequest "http://live.sysinternals.com/Tools" -DisableKeepAlive -UseBasicParsing
#get the links
$links = $sysint.links | Select -Skip 1
#reset counter
$i=0
foreach ($item in $links) {
#download files to subfolder
$uri = "http://live.sysinternals.com$($item.href)"
$i++
$percent = ($i/$links.count) * 100
Write-Verbose "Downloading $uri"
$progParam.status ="SysInternals"
$progParam.currentOperation = $item.innerText
$progParam.PercentComplete = $percent
Write-Progress @progParam
_download -Uri $uri -Path $sub
} #foreach
} #if SysInternals
Write-verbose "Finished $($myinvocation.MyCommand)"
} #end
} #end function
This version has additional parameters that accept pipeline binding by property name, which means you can now run the command like this:
PS C:> Import-csv c:scriptstools.csv | get-mytool2 -path G: -sysinternals
You will need to dot-source this second script to load the function into your session. Otherwise, it works essentially the same. There is one potential drawback to these scripts in that the downloads are all sequential, which means it can take 10 minutes or more to download everything. To build a toolkit even faster, take a look at this alternate approach.
By the way, if you have any favorite troubleshooting or diagnostic tools I hope you’ll let me know. If you can include a direct download link that would be even better.