Skip to content

Latest commit

 

History

History
243 lines (176 loc) · 7.67 KB

README.md

File metadata and controls

243 lines (176 loc) · 7.67 KB
Raw

Coque

Build Status Gem Version

Create, manage, and interop with shell pipelines from Ruby. Like Plumbum, for Ruby, with native (Ruby) code streaming integration.

Installation

Add to your gemfile:

gem 'coque'

Usage

Create Coque commands:

cmd = Coque["echo", "hi"]
# => <Coque::Sh ["echo", "hi"]>

And run them:

res = cmd.run
# => #<Coque::Result:0x007feb5930e408 @out=#<IO:fd 13>, @pid=58688>
res.to_a
# => ["hi"]

Or pipe them:

pipeline = cmd | Coque["wc", "-c"]
# => #<Coque::Pipeline:0x007feb598730b0 @commands=[<Coque::Sh ["echo", "hi"]>, <Coque::Sh ["wc", "-c"]>]>
pipeline.run.to_a
# => ["3"]

Coque can also create "Rb" commands, which integrate Ruby code with streaming, line-wise processing of other commands:

c1 = Coque["printf", '"a\nb\nc\n"']
c2 = Coque.rb { |line| puts line.upcase }
(c1 | c2).run.to_a
# => ["A", "B", "C"]

Rb commands can also take "pre" and "post" blocks

dict = Coque["cat", "/usr/share/dict/words"]
rb_wc = Coque.rb { @lines += 1 }.pre { @lines = 0 }.post { puts @lines }

(dict | rb_wc).run.to_a
# => ["235886"]

Commands can have Stdin, Stdout, and Stderr redirected

(Coque["echo", "hi"] > "/tmp/hi.txt").run.wait
File.read("/tmp/hi.txt")
# => "hi\n"

(Coque["head", "-n", "4"] < "/usr/share/dict/words").run.to_a
# => ["A", "a", "aa", "aal"]

(Coque["cat", "/doesntexist.txt"] >= "/tmp/error.txt").run.wait
File.read("/tmp/error.txt")
# => "cat: /doesntexist.txt: No such file or directory\n"

Coque commands can also be derived from a Coque::Context, which enables changing directory, setting environment variables, and unsetting child env:

c = Coque.context
c["pwd"].run.to_a
# => ["/Users/worace/code/coque"]

Coque.context.chdir("/tmp")["pwd"].run.to_a
# => ["/private/tmp"]

Coque.context.setenv("my_key": "pizza")["echo", "$my_key"].run.to_a
# => ["pizza"]

ENV["my_key"] = "pizza"
Coque["echo", "$my_key"].run.to_a
# => ["pizza"]

Coque.context.disinherit_env["echo", "$my_key"].to_a
# => [""]

Coque also includes a Coque.source helper for feeding Ruby enumerables into shell pipelines:

(Coque.source(1..500) | Coque["wc", "-l"]).run.to_a
# => ["500"]

Asynchrony and Waiting on Processes

Running a Coque command forks a new process, and by default these processes run asynchronously. Calling .run on a Coque command or pipeline returns a Coque::Result object which can be used to get the output (.to_a) or exit code (.exit_code) of the process:

result = Coque['echo', 'hi'].run
# => #<Coque::Result:0x000055da63437838 @out=#<IO:fd 15>, @pid=29236>
puts "its running in the background..."
its running in the background...
result.to_a
# => ["hi"]
result.exit_code
# => 0

However you can also just use .wait to block on a process while it runs:

result = Coque['echo', 'hi'].run.wait
# => #<Coque::Result:0x000055da633c98b0 @exit_code=0, @out=#<IO:fd 17>, @pid=29536>

Or, use .run! to block on the process and raise an exception if it exits with a non-zero response:

Coque["head", "/usr/share/dict/words"].run!
# => nil
Coque["head", "/usr/share/dict/pizza"].run!
# head: cannot open '/usr/share/dict/pizza' for reading: No such file or directory
# RuntimeError: Coque Command Failed: <Coque::Sh head /usr/share/dict/pizza>
from /home/horace/.gem/ruby/2.4.4/gems/coque-0.7.1/lib/coque/runnable.rb:13:in `run!'

There's also a to_a! variant on commands which combines the error handling of run! with the array-slurping of stdout:

Coque['head', '-n 1', '/usr/share/dict/words'].to_a!
=> ["A"]

Coque['head', '-n 1', '/usr/share/dict/asdf'].to_a!
head: cannot open '/usr/share/dict/asdf' for reading: No such file or directory
RuntimeError: Coque Command Failed: <Coque::Sh head -n 1 /usr/share/dict/asdf>
from /code/coque/lib/coque/runnable.rb:11:in `to_a!'

Named (Non-Operator) Method Alternatives

The main piping and redirection methods also include named alternatives:

  • | is aliased to pipe
  • > is aliased to out
  • >= is aliased to err
  • < is aliased to in

So these 2 invocations are equivalent:

(Coque["echo", "hi"] | Coque["wc", "-c"] > STDERR).run!
# is the same as...
Coque["echo", "hi"].pipe(Coque["wc", "-c"]).out(STDERR).run!

Logging

You can set a logger for Coque, which will be used to output messages when commands are executed:

Coque.logger = Logger.new(STDOUT)
(Coque["echo", "hi"] | Coque["wc", "-c"]).run!

Will log:

I, [2019-02-20T20:31:00.325777 #16749]  INFO -- : Executing Coque Command: <Pipeline <Coque::Sh echo hi> | <Coque::Sh wc -c> >
I, [2019-02-20T20:31:00.325971 #16749]  INFO -- : Executing Coque Command: <Coque::Sh echo hi>
I, [2019-02-20T20:31:00.327719 #16749]  INFO -- : Coque Command: <Coque::Sh echo hi> finished in 0.001683 seconds.
I, [2019-02-20T20:31:00.327771 #16749]  INFO -- : Executing Coque Command: <Coque::Sh wc -c>
I, [2019-02-20T20:31:00.329586 #16749]  INFO -- : Coque Command: <Coque::Sh wc -c> finished in 0.001739 seconds.
I, [2019-02-20T20:31:00.329725 #16749]  INFO -- : Coque Command: <Pipeline <Coque::Sh echo hi> | <Coque::Sh wc -c> > finished in 0.003796 seconds.

Streaming Performance

Should be little overhead compared with the equivalent pipeline from a standard shell.

From zsh:

head -c 100000000 /dev/urandom | pv | wc -c
95.4MiB 0:00:06 [14.1MiB/s] [      <=>      ]
 100000000

With coque:

p = Coque["head", "-c", "100000000", "/dev/urandom"] | Coque["pv"] | Coque["wc", "-c"]
p.run.wait
95.4MiB 0:00:06 [14.6MiB/s] [           <=> ]

Development

  • Setup local environment with standard bundle
  • Run tests with rake
  • See code coverage output in coverage/
  • Start a pry console with bin/console
  • Install current dev version with rake install
  • Use rake release to release after bumping lib/coque/version.rb
  • New issues welcome

Further Reading / Prior Art

The concept and API for this library was heavily inspired by Python's excellent Plumbum library.

I relied on many resources to understand Ruby's great facilities for Process creation and manipulation. Some highlights include:

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Coque project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

Building / Releasing

gem build coque.gemspec
gem push coque-<VERSION>.gem
git tag <VERSION>
git push origin <VERSION>