Splitting Large Files in Smaller Parts (Part 1)

PowerShell can split large files in multiple smaller parts, for example to transfer them as email attachments. Today, we focus on splitting files. In our next tip, we show how you can join the parts back together.

To split large files into smaller parts, we created the function Split-File. It works like this:

 
PS> Split-File -Path "C:\Users\tobwe\Downloads\Woman putting gas in Tesla.mp4" -PartSizeBytes 6MB -Verbose
VERBOSE: saving to C:\Users\tobwe\Downloads\Woman tries putting gas in a Tesla.mp4.00.part...
VERBOSE: saving to C:\Users\tobwe\Downloads\Woman tries putting gas in a Tesla.mp4.01.part...
VERBOSE: saving to C:\Users\tobwe\Downloads\Woman tries putting gas in a Tesla.mp4.02.part...
VERBOSE: saving to C:\Users\tobwe\Downloads\Woman tries putting gas in a Tesla.mp4.03.part...

PS C:\> 
 

The -PartSizeByte parameter sets the maximum chunk size, in our example 6MB. When you specify -Verbose, the function shows the part file names while creating them.

To use Split-File, you need to run this code:

function Split-File
{
    
    param
    (
        [Parameter(Mandatory)]
        [String]
        $Path,

        [Int32]
        $PartSizeBytes = 1MB
    )

    try
    {
        # get the path parts to construct the individual part
        # file names:
        $fullBaseName = [IO.Path]::GetFileName($Path)
        $baseName = [IO.Path]::GetFileNameWithoutExtension($Path)
        $parentFolder = [IO.Path]::GetDirectoryName($Path)
        $extension = [IO.Path]::GetExtension($Path)

        # get the original file size and calculate the
        # number of required parts:
        $originalFile = New-Object System.IO.FileInfo($Path)
        $totalChunks = [int]($originalFile.Length / $PartSizeBytes) + 1
        $digitCount = [int][Math]::Log10($totalChunks) + 1

        # read the original file and split into chunks:
        $reader = [IO.File]::OpenRead($Path)
        $count = 0
        $buffer = New-Object Byte[] $PartSizeBytes
        $moreData = $true

        # read chunks until there is no more data
        while($moreData)
        {
            # read a chunk
            $bytesRead = $reader.Read($buffer, 0, $buffer.Length)
            # create the filename for the chunk file
            $chunkFileName = "$parentFolder\$fullBaseName.{0:D$digitCount}.part" -f $count
            Write-Verbose "saving to $chunkFileName..."
            $output = $buffer

            # did we read less than the expected bytes?
            if ($bytesRead -ne $buffer.Length)
            {
                # yes, so there is no more data
                $moreData = $false
                # shrink the output array to the number of bytes
                # actually read:
                $output = New-Object Byte[] $bytesRead
                [Array]::Copy($buffer, $output, $bytesRead)
            }
            # save the read bytes in a new part file
            [IO.File]::WriteAllBytes($chunkFileName, $output)
            # increment the part counter
            ++$count
        }
        # done, close reader
        $reader.Close()
    }
    catch
    {
        throw "Unable to split file ${Path}: $_"
    }
}

Tomorrow, we look into the reverse process: how to take all the chunks and reconstruct the original file.

Learning points for today:

• Use the [IO.Path] class to split file paths
• Use the [IO.File] class to access files content on byte level
• Use Read() to write bytes to a file

psconf.eu – PowerShell Conference EU 2019 – June 4-7, Hannover Germany – visit www.psconf.eu There aren’t too many trainings around for experienced PowerShell scripters where you really still learn something new. But there’s one place you don’t want to miss: PowerShell Conference EU – with 40 renown international speakers including PowerShell team members and MVPs, plus 350 professional and creative PowerShell scripters. Registration is open at www.psconf.eu, and the full 3-track 4-days agenda becomes available soon. Once a year it’s just a smart move to come together, update know-how, learn about security and mitigations, and bring home fresh ideas and authoritative guidance. We’d sure love to see and hear from you!

ReTweet this Tip!