New Async/Streaming Output API.

[iainsmith]

As part of #32 we've talked about adding a new public API for streaming output.

One suggestion would be:

public func shellOut(to command: String,
                     arguments: [String] = [],
                     at path: String = ".",
                     output: (String) -> Void) throws {
   ...
}

/// Usage
shellOut(to: "docker build") { output in
  /// This block will be called multiple times
  print(output)
}

We'd have to check if the output closure would need to be escaping or not. Ideally it wouldn't need to be, but I'm not sure if that is achievable.

What do other people think?

[ghost]

I wanted user input for marathon a while back and I saw this and thought I would press on and make a pull request for ShellOut. I didn't even look here or for existing pull requests, so I've already gone ahead and built a pull request implementing an approach to this.

#34

[iainsmith]

Hey @rob-nash, sorry I missed your comment.

This might be controversial but I want to ask, do we need a top level async API in shellout?

I get that shellout needs to support read_line and other blocking api calls but do we want this to work in a synchronous way?

I'd like to figure out if there is a way this could just work

// my-script
func main() { 
  let answer = try! shellOut("./ask-script")
  print(answer)
}

//ask-script
func main() {
  let answer = readline()
  print(answer)
}

If shell out supported this use case with the existing synchronous api would we want a shellOutAsync method? I’m potentially missing some key use case, so do let me know.

If we do need both sync & async shellout methods then imo we should ensure it's clear in the method name. I also want to suggest that any new API should support streaming output through a closure so that consumers don't need to use a file handler.

Something like:

Sync + streaming output

@discardableResult
public func shellOut(to command: String,
                  arguments: [String] = [],
                        at path: String = ".",
     withOutput output: (String) -> Void) -> throws String {}

// run 1st long running command
try! shellOut("docker compose .") { output in
  // print output as it arrives
  print(output)
}

// second command won't start until first command is finished.
try! shellOut("docker public") { output in
  print(output)
}

// We can still use the existing api for short running commands
let directoryContents = try! shellOut("ls")
print(directoryContents)

Async + streaming shellOutAsync

enum StreamingOutput {
  case text: String
  case errorText: String
  case completed:(Bool) // This could be a throwing function that returns a Bool, or a throwing function that returns all the text
}

public func shellOutAsync(to command: String, 
                  arguments: [String] = [],
                        at path: String = ".",
     withOutput output: @escaping (StreamingOutput) -> Void) {}

shellOutAsync("docker compose .") { output in
  switch output {
    case .text, .errorText(let text)
      print(text)
    case .completed(let success)
     if (success)  { // run next command }
    }
  }
}

Does anyone have an example of when they would use the async version?

[sstadelman]

@iainsmith example of when I would use the async version: I'm doing a long-running pod repo push invocation to automate updating the pod specs for some proprietary frameworks. The curl + compile takes ~1 minute, so it is helpful to get the visual output to confirm things are working correctly. Absent the expected log messages, I would need to just observer the header of Terminal, to see which command is currently executing.

[iainsmith]

Closing as I’ve now migrated to swift-tools-core.