Writing A Compiler For A DSL In Python

I am writing a game in python and have decided to create a DSL for the map data files. I know I could write my own parser with regex, but I am wondering if there are existing pytho

Solution 1:

I've always been impressed by pyparsing. The author, Paul McGuire, is active on the python list/comp.lang.python and has always been very helpful with any queries concerning it.

Solution 2:

Here's an approach that works really well.

abc= ONETHING( ... )
xyz= ANOTHERTHING( ... )
pqr= SOMETHING( this=abc, that=123, more=(xyz,123) )

Declarative. Easy-to-parse.

And...

It's actually Python. A few class declarations and the work is done. The DSL is actually class declarations.

What's important is that a DSL merely creates objects. When you define a DSL, first you have to start with an object model. Later, you put some syntax around that object model. You don't start with syntax, you start with the model.

Solution 3:

Yes, there are many -- too many -- parsing tools, but none in the standard library.

From what what I saw PLY and SPARK are popular. PLY is like yacc, but you do everything in Python because you write your grammar in docstrings.

Personally, I like the concept of parser combinators (taken from functional programming), and I quite like pyparsing: you write your grammar and actions directly in python and it is easy to start with. I ended up producing my own tree node types with actions though, instead of using their default ParserElement type.

Otherwise, you can also use existing declarative language like YAML.

Solution 4:

DSLs are a good thing, so you don't need to defend yourself :-) However, have you considered an internal DSL ? These have so many pros versus external (parsed) DSLs that they're at least worth consideration. Mixing a DSL with the power of the native language really solves lots of the problems for you, and Python is not really bad at internal DSLs, with the with statement handy.

Solution 5:

On the lines of declarative python, I wrote a helper module called 'bpyml' which lets you declare data in python in a more XML structured way without the verbose tags, it can be converted to/from XML too, but is valid python.

https://svn.blender.org/svnroot/bf-blender/trunk/blender/release/scripts/modules/bpyml.py

Example Use http://wiki.blender.org/index.php/User:Ideasman42#Declarative_UI_In_Blender