Random linebreaks in PowerShell standard error output

I think the issue here is that there is a certain width to the console, and the console itself is essentially being redirected to a file.

My solution to this is to redirect the output directly to the pipeline, using:

2>&1 #Interpreted by the console
2>&1 | x #Output directly to x

And then using Out-File with the available -Width parameter:

$(throw thisisnotsometthingyoucanthrowbutisinfactaverylongmessagethatdemonstratesmypoint) 2>&1 |
 Out-File "test.txt" -Width 10000

In this case, powershell will write 10,000 characters before wrapping the text.

However, you also have some odd line breaks in there that I can't replicate right now. That said, now that you know how to send output through the pipeline, you can use other methods to remove the line breaks.

For example, you can use this function which prints out the exact control characters that cause line breaks.

$(throw error) 2>&1 | Out-String | Debug-String

Then, you can go through the output and replace the problem characters, like so:

$(throw error) 2>&1 | Out-String | % {$_ -replace "`r"} | Out-File "test.txt" -Width 10000

Burt Harris' helpful answer shows you one way to avoid the problem, via Start-Process, which requires you to structure the command fundamentally differently, however.

If the output that an equivalent batch file produces is sufficient, there's an easier way: simply call cmd /c and let cmd handle the output redirections, as in your batch file:

cmd /c "`"`"$Env:ProgramFiles\HandBrake\HandBrakeCLI.exe`"`"" @(
    '--input', 'V:\',
    '--title', '1', '--chapter', '1',
    '--start-at', 'duration:110', '--stop-at', 'duration:15',
    '--output', 'pmovie.mp4',
    '--format', 'av_mp4'
    ) '> .\pstd.txt 2> .\perr.txt'

Note how the two output redirections are passed as a single, quoted string, to ensure that they are interpreted by cmd.exe rather than by PowerShell.

Also note the embedded escaped double quotes (`") around the executable path to ensure that cmd.exe sees the entire path as a single, double-quoted string.

As for the extra line breaks you're seeing: I have no specific explanation, but I can tell you how > and 2> work differently in PowerShell - both compared to cmd.exe (batch files) and Start-Process with -RedirectStandard*:

• cmd.exe's redirection operator (>) writes raw bytes to the specified target file, both when redirecting stdout (just > or, explicitly, 1>) and stderr (2>); as such, text output by external programs such as HandBrakeCLI.exe is passed through as-is.

• Start-Process, which uses the .NET API under the hood, does essentially the same when -RedirectStandardOutput and/or -RedirectStandardError parameters are specified.

By contrast, Powershell's own > operator functions differently:

• PowerShell-internally (when calling native PowerShell commands) it converts input objects (that aren't already strings) to strings using PowerShell's rich output formatting system, before sending them to the output file(s), using the character encoding detailed below.

• Output received from external programs is assumed to be text, whose encoding is assumed to be the system's OEM character encoding by default, as reflected in [console]::OutputEncoding and chcp. The decoded text is loaded into .NET strings (which are inherently UTF-16-based) line by line.

  • For redirected stdout output, these strings are re-encoded on output to the target file, using the following encoding by default:

    • Windows PowerShell: UTF-16LE ("Unicode")
    • PowerShell Core: UTF-8 without BOM
    • Note: Only in Windows PowerShell v5.1 or higher and PowerShell Core can you change these defaults - see this answer for details.

  • By contrast, when redirecting stderr output, via stream 2 (PowerShell's error stream), the strings are wrapped in error objects (instances of type [System.Management.Automation.ErrorRecord]) before being output, and the resulting objects are converted to strings based on PowerShell's output-formatting system, and the same character encoding as above is applied on output to the target file.

    • You can see evidence of that in your output containing extra information and lines such as HandBrakeCLI.exe : [10:41:44] hb_init: starting libhb thread and At C:\Test\phandbrake.ps1:1 char:2, ...
    • It also means that extra line breaks can be introduced, because text produced by the output-formatting system assumes a fixed line width based on the console window's width.
    • That said, that doesn't explain the oddly placed line breaks in your case.