Running Background Jobs Efficiently

by Apr 14, 2014

Using background jobs to run tasks concurrently often is not very efficient as you might have seen in a previous tip. Background job performance worsens with the amount of data that is returned by a background job.

A much more efficient way uses in-process tasks. They run as separate threads inside the very same PowerShell, so there is no need to serialize return values.

Here is a sample that runs two processes in the background, and one in the foreground, using PowerShell threads. To create some really long-running tasks, each task uses Start-Sleep in addition to some other command:

$start = Get-Date

$task1 = { Start-Sleep -Seconds 4 Get-Service }
$task2 = { Start-Sleep -Seconds 5 Get-Service }
$task3 = { Start-Sleep -Seconds 3 Get-Service }

# run 2 in separate threads, 1 in the foreground
$thread1 = [PowerShell]::Create()
$job1 = $thread1.AddScript($task1).BeginInvoke()

$thread2 = [PowerShell]::Create()
$job2 = $thread2.AddScript($task2).BeginInvoke()
$result3 = Invoke-Command -ScriptBlock $task3

do { Start-Sleep -Milliseconds 100 } until ($job1.IsCompleted -and $job2.IsCompleted)

$result1 = $thread1.EndInvoke($job1)
$result2 = $thread2.EndInvoke($job2)



$end = Get-Date
Write-Host -ForegroundColor Red ($end - $start).TotalSeconds

Running these three tasks consecutively would take at least 12 seconds for the Start-Sleep statements alone. In reality, the script only takes a bit more than 5 seconds. The result can be found in $result1, $result2, and $result3. In contrast to background jobs, there is almost no time penalty for returning large amounts of data.

Twitter This Tip! ReTweet this Tip!