Oops, seems I haven't updated my blog for 11 months. What I'm doing?

I quit my boring day-job, and have my second daughter, and I've been invited to publish a paper about GNU Artanis on Schemeworkshop2016 affiliated by ICFP (well, I'll have a post for it after the conference), and I've tried many cool projects with my friends of SZDIY Community, and I'm planning to get my hands dirty on Artifitial Intellegence (but I dislike machine learning), and I may have a secret project end of this year (folks, you'll love it)...

But now, I would like to introduce our Lua frontend on GNU Guile, it's not ready for real work, but most of the work is done. There was a bit old, half baked Lua frontend in Guile branch, but seems unmaintain for a long time. I've rewritten one from scratch. Please don't ask me why bother to reinvent wheel. The new one exists now, so let's talk about it. I would like to avoid redundant work in other projects, but not in this one.

The term frontend here doesn't mean the web layout work for a page. A language frontend on Guile is a programming language implemented with Scheme language and taking advantage of all the Guile existing libs/modules, proper tail call, first class continuation, and all the optimizing.

Where is it?

It's still very experimental, but you may get it from github:

git clone git://github.com/NalaGinrut/guile-lua-rebirth.git

For anyone who wants to try, please make sure you have the latest Guile-2.1+ from master branch. It only works for the latest Guile. And it'll display many debug info such as AST analysis, environment printing, before you see the final result. As I said, it's not usable yet. Run it like this:

cd guile-lua-rebirth
guile -L ./

GNU Guile 2.1.3.127-cb421-dirty
Copyright (C) 1995-2016 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,L lua
Happy hacking with Lua!  To switch back, type `,L scheme'.
lua@(guile-user)> 

Now Guile REPL become a Lua REPL. Just try it as you know about Lua.

How does it work?

The most frequent statment I've heard like this: it should be easy to write frontend on Guile, you just parse Lua code and translate it to Scheme. Then let Scheme handle the scope and continuations or blabla. So the noticeable work is almost only the tokenizing and parsing.

NO! It's not how guile-lua-rebirth work, nor does any serious language frontend on Guile. We're not writing an interpreter of Lua with Scheme. We're trying to build a real compiler for Lua, with Scheme. For a multi-language platform like Guile, the Intermediate Language optimizing and code generation has been done. What we have to do is to parse Lua code to AST, and convert the AST to tree-il (the first layer of Guile intermediate representation) without losing any information especially the environment. In addition, we have to handle primitives for Lua, most of them are written in Scheme, and we have to find a proper way to call them from Lua acrossing the modules calling. When most of the work is done, we have to write many libs being compatible with origin Lua as possible to make this language implementation usable. Finally, we have to find a good way to let modules written in Scheme and Lua could be called by each other. Then you may take advantage of the contributions to Guile community.

So, the simplest work is the tokenizing and parsing in our work.

What do we have now?

Guile-lua-rebirth has a well tested lexer (tokenizer), and LALR(1) parser for complete Lua-5.2 grammar, a new designed Abstract Syntax Tree (AST) layer for pre-optimizing, arithmetic operations with paramatric polymorphism (you may plus a number to a string, overloading, if you're familiar with such a term), Lua tables, proper tail call...

Too many obscured terms, huh? Don't worry, they're not so important for a Lua user. Folks just focus on Lua programming, and ignore what the compiler it is. Unless you want to contribute, of course, it's welcome. I'll try to explain how Lua is implemented on Guile in the later posts.

What's new?

In addition, I'm trying to add new features into Lua. The rule to extend is not to break the compatibility as possible. I've added two so far.

Fixed ISSUE-1

The first new feature is actually a fix of Lua language. Let's see the code:

a={b={c={}}}

-- define with colon reference
function a.b.c:foo()
   return self
end

print(a.b.c:foo())
-- ==> table: 0x1a12ba0

print(a.b.c.foo())
-- ==> nil

-----------------------------
-- define with point reference
function a.b.c.bar()
   return self
end

print(a.b.c:bar())
-- ==> nil

print(a.b.c.bar())
-- ==> nil

This is a common issue in Lua. The variable 'a' is a table, which contains another table 'b', and 'b' contains table 'c'. Lua use this approach to mimic Object-Oriented programming: you may treat tables just like classes.

When you define a function in a table, there're two methods to reference the value with a key in the table. The colon reference, and the point reference. The only difference is that colon-ref will pass the current table object as a hidden argument into the function, and bound to the variable named self. Just like this object in Javascript or C++, although they're not the same, they're similar. For point-ref, the self is unbound to any value, so it's nil in default according to Lua spec.

The interesting point is that you have to use colon-ref if you want to get meaningful `self' value. Otherwise, you will get nil, even if you've defined the function with colon. (We name this issue as ISSUE-1 for our later context.)

It is rediculous.

I think it's a simple logical problem. If you define the function with colon, then you must want to reference a meaningful `self' value. If you don't, and you don't want any other people to reference the meaningful `self' value, you'll never define the function with colon, which mentions the compiler pass the current table to the defined function as a hidden argument. On the other hand, if people write a.b.c.foo() in their code, they must want to get a meaningful result of `self', rather than nil. So the problem is that ISSUE-1 prevents people to get meaningful result even if they've defined the function with colon, and make people confuse with code because they can't figure out where's the error when they encoutered an unexpected nil somewhere.

My idea? I dropped the old design to pass the current table as the hidden argument. Because such an implementation will cause ISSUE-1. What I did in guile-lua-rebirth is to bind `self' to the current table in the nearest environment of the defined function. This makes `self' get a solid value when the function is defined with colon. No matter how you reference the `self' with colon or point.

---- The new design in guile-lua-rebirth
a={b={c={}}}

-- define with colon reference
function a.b.c:foo()
   return self
end

print(a.b.c:foo())
-- ==> table: 0x1a12ba0

print(a.b.c.foo())
-- ==> table: 0x1a12ba0

Of course, the activity is unchanged when you defined the function with point, the `self' will be nil either. So it is unnecessary to show the redundant code here.

Added `continue' statement

Many people thought Lua is just similar to C, since most of the syntax looks like C. Well, it is not true. At least, you can't use `continue'. You may read the discussion on StackOverflow. And the explain from the Lua author:

This is a common complaint. The Lua authors felt that continue was only one of a number of possible new control flow mechanisms
 (the fact that it cannot work with the scope rules of repeat/until was a secondary factor.)
In Lua 5.2, there is a goto statement which can be easily used to do the same job.

There's no `continue' statement in Lua, but this feature is useful in certain context. For example, you want to print all odd number from 1 to 10.

---- For Lua-5.1
for i=1,10 do
  if i%2~=0 then print(i) end
end
---- You have to change code logic to make it work



---- For Lua-5.2+ or Lua-jit
for i=1,10 do
  if i % 2 == 0 then goto continue end
  print(i)
  ::continue::
end
---- You have to use GOTO which is obsolete for modern language



---- For guile-lua-rebirth
for i=1,10 do
  if i%2==0 then
     print(i)
  else
     continue
  end
end
---- No need to explain for any C user

Guile-lua-rebirth hasn't implemented GOTO yet. It won't be too difficult to implement when we have delimited-continuations. But I'm still thinking if we really need it, except for compatibility with Lua community. After all, Edsger Dijkstra has the famous paper GOTO statement considered harmful. But folks may argue against as "continuations are GOTOs too, and you plan to add it".

Well, I don't know.

I have to close comment since I found my blog is unstable to get comments from folks. It's all my bad, I should build a better blog. And I saw comments from you for encouraging me. Thank you very much Hayden Jones!