Mapping file descriptors of pwsh process to Powershell streams #7989
Comments
|
Alternately, the streams should just be serialized to the file descriptor corresponding to the stream numbers documented here, and anyone who wants to see progress, warning etc. output on stdout can just manually redirect all of those file descriptors into |
|
PowerShell can redirect streams https://blogs.technet.microsoft.com/heyscriptingguy/2014/03/30/understanding-streams-redirection-and-write-host-in-powershell/ |
|
Yeah, you'll need to redirect the streams from within a PS session. There's not a complete 1:1 mapping of all PS streams to other console streams, so you'll have to perform this redirection when calling from a PS shell or from within the script itself to be sure PowerShell is actually handling the redirections. Otherwise, I believe most of these will get squashed into the output / stderr stream(s) by default. |
|
@vexx32 I don't always have access to the script though (nor to the scripts that it transitively calls). From my perspective I'm just trying to evaluate some black box binary which emits data to a variety of output streams, and I only care about retrieving one of them, which is trivial with other shells and script interpreters. If some functions somewhere do |
|
PowerShell has more streams than STDOUT and STDERR, but these are only understood from within PowerShell. By default, I believe it is expected for the extended streams to be sent to STDOUT from pwsh.exe as it can't send it anywhere else. Trying to understand the use case here. If you don't want the Warning stream, you could just add Are you asking for something like: |
|
@SteveL-MSFT Processes also have more file descriptors than just The use case is to have a script that uses various modules (e.g. PowerCLI) and emits a fixed format. It's going to live alongside a variety of other scripts (most of which are not implemented in Powershell) that emit an identical format. Right now the script pollutes stdout with warning/progress/debug information emitted by the various modules it uses. If this was instead a bash script, stuff that is emitted on fd 3 by a sourced third party script or exe would not contaminate the output of fd 1 (i.e. stdout). I can certainly try and suppress everything besides the Powershell output stream at a global level, but I'd prefer to still have this information available if needed. More importantly, I'd like to avoid having to modify the script at all. |
Yes, something like that would be very useful, provided it can be specified exhaustively for all Powershell streams. |
|
Arguably, what PowerShell should always have done is to map all streams other than the success stream to stderr (this is basically what native shells / utilities are forced to do: anything that's not data must go to stderr) - throwing everything into stdout was never a good idea. It is another unfortunate example of PowerShell treating the outside world as an afterthought - which was less of a problem on Windows, but Unix users will expect conformant behavior. It would be a breaking change, @SteveL-MSFT, but perhaps it's not too late for Core? +1 for the option to capture streams individually. |
|
P.S.: While everything currently goes to stdout in the absence of redirections or in the presence of only stdout redirection ( Here are variations of a simple test command that exercises all streams: # From Bash:
$ pwsh -nop -command "write-output 1; write-error 2; write-warning 3; write-verbose -vb 4; \$DebugPreference = 'Continue'; Write-Debug 5; Write-Information -InformationAction continue 6; Write-Host host"
# From cmd.exe
C:> pwsh -nop -command "write-output 1; write-error 2; write-warning 3; write-verbose -vb 4; $DebugPreference = 'Continue'; Write-Debug 5; Write-Information -InformationAction continue 6; Write-Host host"
# From cmd.exe, Windows PowerShell
C:> powershell -nop -command "write-output 1; write-error 2; write-warning 3; write-verbose -vb 4; $DebugPreference = 'Continue'; Write-Debug 5; Write-Information -InformationAction continue 6; Write-Host host" |
|
@mklement0 changing from STDOUT to STDERR is probably too big a breaking change to take at this point, adding a switch to |
|
@SteveL-MSFT, I just remembered the discussion about providing a separate Perhaps we should consider fixing all the broken things there, to provide a CLI that fits better into the Unix world and has saner defaults overall (perhaps with a different name than Aside from not loading
|
|
@mklement0 that may be a good solution as it wouldn't be a breaking change, but then |
|
I just came across this thread as I wanted to capture warning/debug/verbose streams separately too. Turns out you can run pwsh with `pwsh -o XML -c "Write-Verbose 123" which pipes XML based messages to stdout and stderr with all of the information (and more) that you can use to capture all output. I have almost managed to make it work properly, my current problem is incorrect order of incoming data e.g. from below script: $ErrorActionPreference = "Continue"
$DebugPreference = "Continue"
$VerbosePreference = "Continue"
$PSVersionTable | Out-Host
Write-Debug "Hello DEBUG"
Write-Verbose "Hello VERBOSE"
Write-Output "Hello OUTPUT"
Write-Host "Hello HOST"
Write-Information "Hello INFORMATION"
Write-Warning "Hello WARNING"
Write-Error "Hello ERROR"
throw "Hello THROW"I see I am using node.js to process the output: const child_process = require('child_process');
const fs = require('fs');
const path = require('path');
const util = require('util');
const Saxophone = require('saxophone');
const cp = child_process.spawn(
'pwsh',
['-NoProfile', '-NoLogo', '-NonInteractive', '-OutputFormat', 'XML', '-File', path.join(__dirname, 'script.ps1')],
);
new CliXMLParser(cp.stdout, 'stdout');
new CliXMLParser(cp.stderr, 'stderr');
function CliXMLParser(stream, name) {
this.stream = stream;
this.name = name;
this.currentTag = null;
this.debugOut = fs.createWriteStream(util.format('%s.xml', name), 'utf-8')
const self = this;
this.stream.on('data', (chunk) => {
const text = chunk.toString('utf-8');
console.log('chunk received', text.slice(0, 100));
if (text.indexOf('#< CLIXML') === 0) {
self.parser = new Saxophone();
self.parser.on('tagopen', tag => {
self.currentTag = tag;
})
self.parser.on('tagclosed', () => {
self.currentTag = null;
})
self.parser.on('text', (text) => {
if (!self.currentTag) {
return;
}
const attributes = Saxophone.parseAttrs(self.currentTag.attrs);
if (text.contents === '_x000A_') {
return; // process.stdout.write("\n");
}
if(self.currentTag.name === 'ToString') {
console.log(text.contents)
}
if(attributes.S && attributes.S !== "Output") {
process.stdout.write(util.format('##%s ', attributes.S.toLowerCase()))
}
if(self.currentTag.name === 'S') {
console.log(text.contents)
}
})
self.parser.on('finish', () => {
console.log('Parsing finished.');
});
self.parser.on('error', (err) => {
console.error(err);
});
} else {
self.debugOut.write(chunk);
if (self.parser){
self.parser.write(chunk);
}
}
})
this.stream.on('error', err => {
console.error(err);
})
this.stream.on('end', () => {
console.log("Closing debug out")
self.debugOut.close();
})
}
cp.on('exit', (code) => {
console.log("Pwsh exited with code %d", code);
})
console.log("Started pwsh process: %s", cp.pid); |
|
You may have hit a bug, which I've just reported in #12489. Also, it seems that host output is captured as success-stream output ( |
Summary
I need a way to specify (using command line arguments to
pwsh) a mapping between Powershell streams and process file descriptors . Right now I can't see a way to prevent warnings, progress information, and debug messages from being munged together on stdout alongside data that needs to be in a particular format (e.g. JSON).Steps to reproduce
Put the following code into a file
test:In some shell of your choosing, run the equivalent of:
Expected behavior
The file
outputshould contain:Actual behavior
The file
outputcontains:Environment data
The text was updated successfully, but these errors were encountered: