Samson's machete

Oba pipeline style!

Posted by NalaGinrut on 25 April 2014 4:51 PM (scheme | guile | python | pipeline | programming)

The concept of pipeline in functional programming means a succession of functions that operate, one after another, on an array of data, which consists of a chain of processing elements arranged so that the output of each element is the input of the next. One of the famous practices is Unix pipeline.

This article is about pipeline-like programming pattern, rather than 'yet another tutorial of pipeline tools'.

Let's see some code:

In Python

Here's a silly code as an example.

def get_number():
    return input("Give me a number: ")

def num_filter(num):
    print "oh you input %d!" % num
    return num % 2

def word_picker(hit):
    print "checking..."
    return ('even', 'odd')[hit]

def result_show(result):
    print "It is an %s number." % result

result_show(word_picker(num_filter(get_number)))

The process is very simple, get a number, and check if it's even or odd, finally it print out the result.

But this code has a problem, each time you want to add a level in your cascaded functions chain, you have to modify the code. The worst case is when you have more than 10 levels in your functions calling chain, you can't write it as a chain. You have to split them like this:

num = get_number()
hit = num_filter(num)
result = word_picker(hit)
result_show(result)

This could be clearer, and it's easy to add a 'add_one' step after 'get_number', which plus 1 to the number you just typed. But you've added a lot of code and several temprorary variables. Let's hope your compiler can eliminate all the redundant part for you during the optimization.

Anyway, it's better not to modify the original code when you want to add new code.

Now here is the pipeline one:

def make_pipeline(procs):
    return lambda x: reduce(lambda y,p:p(y), procs, x)

procs = [num_filter, word_picker, result_show]
f = make_pipeline(procs)
f(get_number())

def add_one(x):
    return 1+x

f2 = make_pipeline(procs.insert(0, add_one))
f2(get_number())

In Scheme

Although many pythoners enjoy lambda and reduce, not all the people have realized that these stuffs are borrowed from Lisp/Scheme land. There're cooler things in Scheme, maybe we should drop reduce and give others a try, say, fold:

(use-modules (srfi srfi-1)) ; `fold' dwells srfi-1

(define (get-number)
  (display "Give me a number: ")
  (read))

(define (num-filter num)
  (format #t "oh you input ~d!~%" num)
  ;; ~d is similar to %d in Python, and ~% means newline
  (modulo num 2))

(define (word-picker hit)
  (display "checking...\n")
  (list-ref '("even" "odd") hit)) ; list accessing in Scheme

(define (result-show result)
  (format #t "It is an ~a number.~%" result))
  ;; ~a is similar to %r in Python

(define (make-pipeline . procs)
  (lambda (x) (fold (lambda (y p) (y p)) x procs)))

(define f (make-pipeline num-filter word-picker result-show))

(f (get-number))

See, there's no much difference from Python code. And you may read the manual of fold here.

After all, the content of this article is nothing but introduced a possible way to implement compose, which is very useful and interesting in programming. Now you already know what it is and how it can be, maybe it's time to hack more code, huh?

9 responses


  1. amz3 says:

    This is similar to bash pipe ``|``.


  2. Nala Ginrut says:

    @amz3 Yes, it's the same principle. ;-)


  3. essay writing service reviews says:

    The article about Oba pipeline style is very nice. This is my first time that I am hearing about Oba pipeline style. Thank you for your information.


  4. Who Can Do My Homework says:

    You are indicate the subject and all around clarified each point superbly. I like the way you have composed this article. Much obliged to you such a great amount for sharing. I generally cherish educational post which helps us to expand our insight.


  5. Do my paper says:

    You are show the subject and all around elucidated each point sublimely. I like the way you have formed this article. Thankful to you such an awesome sum for sharing. I for the most part love instructive post which helps us to extend our knowledge.


  6. Do my paper says:

    You are show the subject and all around elucidated each point sublimely. I like the way you have formed this article. Thankful to you such an awesome sum for sharing. I for the most part love instructive post which helps us to extend our knowledge.


  7. corporate video production says:

    In numerous applications there is a requirement for different procedures to speak with each other - trading information or control data. This should be possible by different strategies like Pipes, signs, shared memory, message lines and so forth.


  8. coursework help uk says:

    Pipelines can be made under program control. The Unix pipe system call requests that the working system develops another unknown pipe protest. This outcome in two new opened record descriptors all the while: the readjust end of the pipe and the compose just end. The pipe closes give off an impression of being ordinary, mysterious record descriptors, with the exception of that they have no ability to seek.


  9. dissertation editing help says:

    Pipeline Style programming brings a more useful programming style of coding to C#. Enlivened by the F# pipe administrator and LINQ technique punctuation, this small arrangement of augmentation capacities, add more useful programming elements, utilizing combinators.

Leave a Reply