PowerShell

Building a PowerShell Troubleshooting Toolkit Revisited

Recently we posted an article about a PowerShell script you could use to build a USB key that contains free troubleshooting and diagnostic utilities. Those scripts relied on a list of links that were processed sequentially by the Invoke-Webrequest cmdlet. The potential downside is that it might take a bit of time to completely download everything. Wouldn’t it be much nicer if we you could download files, let’s say in batches of five?

Unfortunately, there are no cmdlets or parameters that you can use to throttle a set of commands. You could try to create some sort of throttling mechanism with PowerShell’s job infrastructure. If you are proficient with .NET, then you could try your hand at runspaces and runspace pools. Frankly, those make my head hurt, and I don’t expect IT pros to have to be .NET developers to use PowerShell. Fortunately, there is an alternative that I think is a good compromise between usability and systems programming: workflow.

PowerShell 3.0 brought us the ability to create workflows in PowerShell script. The premise of a workflow is that you can orchestrate a series of activities that can run unattended on 10, 100, or 1,000 remote computers. I don’t have space here to fully explain workflows. There is a chapter on workflow in the PowerShell in Depth book from Manning. But one of the great features in my opinion is the ability to execute multiple commands simultaneously.

In a workflow, you can use a ForEach construct with the –Parallel parameter.

Foreach -parallel ($item in $items) {
do-something $item
}

The –Parallel parameter only works in a workflow. If there are 10 items, then the Do-Something command will run all 10 at the same time. Or you can throttle the number of simultaneous commands.
Foreach –parallel –throttle 5 ($item in $items) {
do-something $item
}

Sponsored Content

What is “Inside Microsoft Teams”?

“Inside Microsoft Teams” is a webcast series, now in Season 4 for IT pros hosted by Microsoft Product Manager, Stephen Rose. Stephen & his guests comprised of customers, partners, and real-world experts share best practices of planning, deploying, adopting, managing, and securing Teams. You can watch any episode at your convenience, find resources, blogs, reviews of accessories certified for Teams, bonus clips, and information regarding upcoming live broadcasts. Our next episode, “Polaris Inc., and Microsoft Teams- Reinventing how we work and play” will be airing on Oct. 28th from 10-11am PST.

Now only five commands will run at a time. As one command finishes, the next one in the queue is processed. You don’t have to write any complicated .NET code to take advantage of this feature. Use a workflow. And there’s no rule that says I have to use a workflow with a remote computer. I can create a workflow and run it locally, which is what I’ve done with my original script.

#requires -version 4.0

#create a USB tool drive using a PowerShell Workflow

Workflow Get-MyToolsWF {
<#

.Synopsis
Download tools from the Internet.
.Description
This PowerShell workflow will download a set of troubleshooting and diagnostic tools from the Internet. The path will typically be a USB thumbdrive. If you use the -Sysinternals parameter, then all of the SysInternals utilities will also be downloaded to a subfolder called Sysinternals under the given path.

You can limit the number of concurrent downloads with the ThrottleLimit parameter which has a default value of 5.
.Example
PS C:> Get-MyToolsWF -path G: -sysinternals

Download all tools from the web and the Sysinternals utilities. Save to drive G:.
.Notes
Last Updated: September 30, 2014
Version     : 1.0

  ****************************************************************
  * DO NOT USE IN A PRODUCTION ENVIRONMENT UNTIL YOU HAVE TESTED *
  * THOROUGHLY IN A LAB ENVIRONMENT. USE AT YOUR OWN RISK.  IF   *
  * YOU DO NOT UNDERSTAND WHAT THIS SCRIPT DOES OR HOW IT WORKS, *
  * DO NOT USE IT OUTSIDE OF A SECURE, TEST SETTING.             *
  ****************************************************************

.Link
Invoke-WebRequest

#>

[cmdletbinding()]
Param(
[Parameter(Position=0,Mandatory=$True,HelpMessage="Enter the download path")]
[ValidateScript({Test-Path $_})]
[string]$Path,
[switch]$Sysinternals,
[int]$ThrottleLimit=5
)

Write-Verbose -Message "$(Get-Date) Starting $($workflowcommandname)"

Function _download {
    [cmdletbinding()]
    param([string]$Uri,[string]$Path)

    $out = Join-path -path $path -child (split-path $uri -Leaf)

    Write-Verbose -Message  "Downloading $uri to $out"

    #hash table of parameters to splat to Invoke-Webrequest
    $paramHash = @{
     UseBasicParsing = $True
     Uri = $uri
     OutFile = $out
     DisableKeepAlive = $True
     ErrorAction = "Stop"
    }

    Try {
       Invoke-WebRequest @paramHash
       Get-Item -Path $out
        }
    Catch {
        Write-Warning -Message "Failed to download $uri. $($_.exception.message)"
    }
    } #end function

Sequence {

<#
csv data of downloads
The product should be a name or description of the tool.
The URI is a direct download link. The link must end in the executable file name (or zip or msi).
The file will be downloaded and saved locally using the last part of the URI.
#>

$csv = @"
product,uri
HouseCallx64,http://go.trendmicro.com/housecall8/HousecallLauncher64.exe
HouseCallx32,http://go.trendmicro.com/housecall8/HousecallLauncher.exe
"RootKit Buster x32",http://files.trendmicro.com/products/rootkitbuster/RootkitBusterV5.0-1180.exe
"Rootkit Buster x64",http://files.trendmicro.com/products/rootkitbuster/RootkitBusterV5.0-1180x64.exe
RUBotted,http://files.trendmicro.com/products/rubotted/RUBottedSetup.exe
"Hijack This",http://go.trendmicro.com/free-tools/hijackthis/HiJackThis.exe
WireSharkx64,http://wiresharkdownloads.riverbed.com/wireshark/win64/Wireshark-win64-1.12.1.exe
WireSharkx32,http://wiresharkdownloads.riverbed.com/wireshark/win32/Wireshark-win32-1.12.1.exe
"WireShark Portable",http://wiresharkdownloads.riverbed.com/wireshark/win32/WiresharkPortable-1.12.1.paf.exe
SpyBot,http://spybotupdates.com/files/spybot-2.4.exe
CCleaner,http://download.piriform.com/ccsetup418.exe
"Malware Bytes",http://data-cdn.mbamupdates.com/v2/mbam/consumer/data/mbam-setup-2.0.2.1012.exe
"Emisoft Emergency Kit",http://download11.emsisoft.com/EmsisoftEmergencyKit.zip
"Avast! Free AV",http://files.avast.com/iavs5x/avast_free_antivirus_setup.exe
"McAfee Stinger x32",http://downloadcenter.mcafee.com/products/mcafee-avert/Stinger/stinger32.exe
"McAfee Stinger x64",http://downloadcenter.mcafee.com/products/mcafee-avert/Stinger/stinger64.exe
"Microsoft Fixit Portable",http://download.microsoft.com/download/E/2/3/E237A32D-E0A9-4863-B864-9E820C1C6F9A/MicrosoftFixit-portable.exe
"Cain and Abel",http://www.oxid.it/downloads/ca_setup.exe
"@

#convert CSV data into objects
#save converted objects to a variable seen in the entire workflow
$workflow:download  = $csv | ConvertFrom-Csv

} #end sequence


Sequence {

foreach -parallel -throttle $ThrottleLimit ($item in $download) {

    Write-Verbose -message "Downloading $($item.product)"
    _download -Uri $item.uri -Path $path

} #foreach item


} #end sequence


Sequence {
#region SysInternals

if ($Sysinternals) {
    #test if subfolder exists and create it if missing
    $subfolder = Join-Path -Path $path -ChildPath Sysinternals

    if (-Not (Test-Path -Path $subfolder)) {
        New-item -ItemType Directory -Path $subfolder
    }

    #get the page
    $sysint = Invoke-WebRequest "http://live.sysinternals.com/Tools" -DisableKeepAlive -UseBasicParsing

    #get the links
    $links = $sysint.links | Select -Skip 1

    foreach -parallel -throttle $ThrottleLimit ($item in $links) {
     #download files to subfolder
     $uri = "http://live.sysinternals.com$($item.href)"
     Write-Verbose -message "Downloading $uri"

     _download -Uri $uri -Path $subfolder

    } #foreach
} #if SysInternals

} #end sequence

Write-verbose -message "$(Get-Date) Finished $($workflowcommandname)"

} #end workflow

There are rules for a workflow so I had to make a few adjustments. Workflows aren’t designed to run from pipelined input so the CSV data is embedded in the script. Otherwise, you should recognize most of the code. The key difference is that downloads now happen in parallel and throttled.
foreach -parallel -throttle $ThrottleLimit ($item in $download) {
  Write-Verbose -message "Downloading $($item.product)"
  _download -Uri $item.uri -Path $path
} #foreach item

I do the same thing for the SysInternals links.
foreach -parallel -throttle $ThrottleLimit ($item in $links) {
 #download files to subfolder
 $uri = "http://live.sysinternals.com$($item.href)"
 Write-Verbose -message "Downloading $uri"
 _download -Uri $uri -Path $subfolder
} #foreach


Once the workflow is loaded into my session, which you can do by dot-sourcing the script, I can run it the same way I would any command.

PS C:> Get-MyToolsWF -path G: -sysinternals

Use the –Verbose cmdlet if you want to see more detail. Using the workflow and parallel processing, I can download everything in about five minutes! Personally, I don’t find much use for workflows, but I love this parallel feature and hope that someday we’ll see it as part of the main PowerShell language.

Related Topics:

BECOME A PETRI MEMBER:

Don't have a login but want to join the conversation? Sign up for a Petri Account

Register
Comments (0)

Leave a Reply

External Sharing and Guest User Access in Microsoft 365 and Teams

This eBook will dive into policy considerations you need to make when creating and managing guest user access to your Teams network, as well as the different layers of guest access and the common challenges that accompany a more complicated Microsoft 365 infrastructure.

You will learn:

  • Who should be allowed to be invited as a guest?
  • What type of guests should be able to access files in SharePoint and OneDrive?
  • How should guests be offboarded?
  • How should you determine who has access to sensitive information in your environment?

Sponsored by: