Reading HTTP Headers from Websites

by Sep 8, 2021

When you navigate to a web page, your browser silently receives meta information inside the HTTP header which is typically invisible.

To display the HTTP header information for any website, try this:

# replace URL with any web page you like:
$url = 'www.tagesschau.de'
(Invoke-WebRequest -Method Head -Uri $url -UseBasicParsing).RawContent

The trick is to use the -Method parameter and submit the “Head” value to get the header information instead of the web page content.

The result looks similar to this:

 
HTTP/1.1 200 OK
Connection: keep-alive
Access-Control-Allow-Origin: *
Cache-Control: max-age=14
Content-Type: text/html; charset=utf-8
Date: Tue, 10 Aug 2021 12:06:37 GMT   
 

The header info returned by a web page can vary greatly. When you replace the URL in above example with ‘www.google.de’, for example, you see many more instructions being sent to your browser, and you can actually “see” how the web page instructs your browser to set new cookies – so you can check whether the web page sets cookies before asking for consent or not:

 
HTTP/1.1 200 OK
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Transfer-Encoding: chunked
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Date: Tue, 10 Aug 2021 12:07:06 GMT
Expires: Tue, 10 Aug 2021 12:07:06 GMT
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Set-Cookie: NID=221=C8RdXG_2bB_MwOG33_lS0hx3P5TF5_vaamoT0xj3yKgZPzCjr_g70NvJXkhenV_GMt1TYYU6XwmNOtlkRKRADiXgYJWWYp671M3DFL8DCxM_J1Cl01r39-jfA7sIxu1C
-0B7CHI-8WfXj5IGZ5dHtRxndNA84cpQov5phLhi7l8; expires=Wed, 09-Feb-2022 12:07:06 GMT; path=/; domain=.google.de; HttpOnly
Server: gws   
 

If you like you can get the headers in tabular form as well – simply replace RawContent by Headers:

# replace URL with any web page you like:
$url = 'www.google.de'
(Invoke-WebRequest -Method Head -Uri $url -UseBasicParsing).Headers


Twitter This Tip! ReTweet this Tip!