Downloading Information from Internet (Part 1)

by Apr 13, 2018

PowerShell comes with two powerful cmdlets that you can use to retrieve information from the internet. Today, we focus on Invoke-WebRequest.

This cmdlet serves as a simple web client. Pass it a URL, and it downloads the webpage for you. These simple lines download the agenda for you:

$page = Invoke-WebRequest -Uri -UseBasicParsing 

Since it is in JSON format, pipe it to ConvertFrom-Json to get objects:

$page = Invoke-WebRequest -Uri -UseBasicParsing 
$page.Content | ConvertFrom-Json | Out-GridView

However, sometimes (like in this example), it does not correctly “unroll” the collection results, so do it manually:

$page = Invoke-WebRequest -Uri -UseBasicParsing 
$($page.Content | ConvertFrom-Json) | Out-GridView

This way, you can download whatever information is stored on a webpage, and process it. If a web page delivers XML, you could convert it to XML, and if it is plain HTML, you could use regular expressions to pick the information you are after. More about this in a future tip.

Twitter This Tip! ReTweet this Tip!