Monday, August 30, 2010

Self compiling!


I am pleased to say that bmx2 (STILL haven't decided on a new name yet) is now fully self-compiling!

In theory, this just involved converting the existing bmx source files to bmx2 via a little 'convbmx' util, but in practice it has been a tricky and at times somewhat mind bending exercise.

In particular, with 2 sets of sources I had to stop and think several times which one I actually needed to modify - and occasionally even found myself modifying the wrong one!

But it all seems to be working remarkably well now, and although I've only managed to compile the compiler itself as a C++ target (because that's the only target with sufficient filesystem support right now), the 'new' compiler has compiled all my other little tests successfully to html5, flash, xna and android.

Some more details about how it all works...

Bmx2 will required that the appropriate SDK's for the various targets be installed separately. For example, to compile to c++ you'll need MinGW installed. Current list of supported targets/SDK's is:

* HTML5 - HTML5 compatible browser required for testing.
* Flash - Open source Flex SDK required.
* XNA - Visual C# Express + XNA framework
* Android - The Android SDK (plus Java SDK and 'Ant' on Windows).
* Native (ie: C++) - Mingw/GCC (will probably add visualc support too in future).
* iPhone - XCode.

This has the obvious benefit of making life a whole bunch easier for myself, but it also means you'll be able to take full advantage of the various SDK tools available. For example, iPhone projects will be 'real' xcode projects, so you'll be able to edit them in xcode, mess with the NIB file etc. When you compile your bmx2 program, the translator simply replaces a chunk of text in a 'main' file somewhere in the target project, and leaves the rest of the project alone.

Installation of SDK's can be a little messy in some cases, but the compiler already automagically deals with any PATH or ENV var issues which makes life a whole lot easier.

Some more details on the new language:

* 'Type' renamed to 'Class'.

* Identifiers now case sensitive.

* Private can at last be used inside classes!

* A single 'End' can be used to terminate blocks, eg:

Function MyFunc()
Print "Boo!"

Ye olde bmx1 style terminators are still permitted too.

* Function/method overloading added, eg:

Function DrawImage( image:Image,x,y )

Function DrawImage( image:Image,x,y,frame )

* 'Proper' constructors, eg:

Method New( x,y )

* Support for 'properties', eg:

Method X() Property
Return x

Method X( x ) property

* Simple class template support, eg:

Local list:=New List<Actor>

* Simple 'boxing', eg:

Local list:=New List<IntObject>
list.AddLast 10
list.AddLast 20
Local n=list.Last()

* Auto type inference for initializers, eg:

Local t:=New Actor 'instead of Local t:Actor=New Actor

* Strict mode is back, but will effectively mean bmx1-style SuperStrict (or possibly even stricter). Default mode will be plain bmx1-style Strict (and probably even lazier).

I agonized for a while over whether bmx2 should go 'SuperStrict only', but after realizing that some of the superstrict crowd wanted things even stricter(!) while I wanted things more relaxed, I decided to keep Strict in there in the hope the final product will appeal to a wider range of people.

I have a related theory on this: SuperStrict users prefer C# over C++; Strict users vice versa...

* Module 'Imports' can now be cyclic, eg:

'Module A
Import B

'Module B
Import A

I can't BELIEVE I thought this wasn't a big deal in bmx1!

It was kind of mission to get going too - I basically had to change the compilation process from a top down, 2 pass algorithm to a 'compile on demand' one.

It worked out really well though - the entire program is compiled simply by compiling 'Main'...which causes anything main depends on to compile, and so on.

As a nice side effect, you end up with a very clear picture of what code the program actually uses, meaning apps should be nice and tight.

It's not all good news though (assuming you already though it was) - here are a few nasty 'detail' things that may or may not be fixed/improved over time, but for now would be very hard to do efficiently in ALL target languages. I consider them pretty minor, but you're free to disagree with me on that...

* Strings and Arrays are NOT objects in bmx2 the way they are in bmx1 - they are more like primitives such as int and float, eg:

Local t:Object="Hello world" 'OK in bmx1, Error in bmx2.

* Multidimensional arrays are out, although you can still have arrays of arrays, eg:

Local t[10,20] 'OK in bmx1, Error in bmx2
Local t[][10] 'OK in bmx1 and bmx2.

* Arrays can not be 'downcast' in bmx2, eg:

Local t:Object[]=New Thing[10]
Local p:Thing[]=Thing[]( t ) 'OK in bmx1, Error in bmx2.

This will mean that arrays may have to be manually 'wrapped' in some situations.

And that's it for now. Hope you found it informative!



  1. Hi,
    first: thank you for the update and for the info about BMX2 and how it works!

    * Identifiers now case sensitive. --> damn!! I know I'll spedt half of my time checking my typo-errors...

    Type now are called Class: some 'special' reason to change the name?

    You wrote about 'cyclic import' and other things: do you think some of these can be 'ported' to BMX in a far far future or they will be an 'exclusive' of BMX2?

    I cant' wait anymore!!!

    Thank you very much again!

  2. Sounds really great!
    Call me easily pleased but I like the 'End' only implementation

    1. Are you planning on supporting Boolean?
    2. What happens if you do not explicitly return something in a function? I know Java complains

    Function State(s)
    If s=1 Then Return True

    As above, if s is not 1 then no other explicit value is returned


  3. Woohoo so when can we get our hands on it?

  4. It's all coming together! I can't wait to see it in action and start playing with it. I'm not too bothered about the little differences actually. All languages have their quirks. Cross-platform development is usually full of little compromises.

  5. Sounds Awesome! Can't wait to try it out.

    * Identifiers now case sensitive.
    NOOOOO!!!!! Say it aint so!!
    I understand that most of the target languages are that way, but why force it before translation?

  6. I bet it's because of javascript (which has no static typing). I was working on a language on top of javascript, and to make it case-insensitive you about have to write your own type system on top of javascript (see for example closure compiler and how it uses jsdocs for type information).

  7. An update, great. As many have said due to it's complex nature comprises need to be made, I'll miss multi-dim arrays but I'll get by.

    Now when can we expect release or beta? I am itching to get an andriod phone but cannot justify it yet, but BMX2 will change that.

  8. Hey mark,
    thank you for being so informative. I always have a blast reading your blog entries and I'm really, really, really looking forward to bmx2 (really!)!

    I'm currently working on my HTML5 skills but it seems that I can take it easy because your language will be absolutely what I need :)


  9. Nice. Any thoughts on 3rd party IDE's?

  10. "Any thoughts on 3rd party IDE's? "

    Check Ziggy's blog:

  11. Thanks. While I applaud the effort going into BLIde, that wasn't exactly what I had in mind.

    Not that it isn't a good IDE, it's certainly better value for money than, say VisualStudio.

    But it's still a far cry from IntelliJ, Eclipse or NetBeans, and as far as I'm aware (but please feel free to correct me if I'm out of date on this) still lacks fundamental functionality you grow accustomed to when working with (say) IntelliJ.

    Things like Refactoring and Code Analysis, which can really help on large, complex projects.

  12. Also quick follow-up question, my main interest is this is for Android development, are there any plans for allowing an Android developer to import proper Java classes into BMX2, provided one accepts that one is tying oneself to a specific platform?

  13. I just got pointed to this blog and all I have to say is THANK YOU! I unwillingly left Blitz almost 2 years ago because I wanted to put my games online in a browser so I went to Java and Flash, but I absolutely hated the enviroments, IDE's and the languages and missed Blitz everyday. With this announcement, I will be coming back to Blitz.

    Thank you!

    Oh, any expected release dates?

  14. Hi,

    > Type now are called Class: some 'special' reason to change the name?

    I just prefer 'Class'.

    There's a 'koan' in the Java docs:

    Variables have type
    Objects have class

    Which I find a useful distinction, esp. as a compiler writer!

    > You wrote about 'cyclic import' and other things: do you think some of these can be 'ported' to BMX

    Probably not.

    > Are you planning on supporting Boolean?

    Type 'Bool' is in there, but it behaves very much like Int...

    > Any thoughts on 3rd party IDE's?

    Ziggy's working on a version of Blide, but I suspect he's hating me for the type-inference stuff!

    > Things like Refactoring and Code Analysis, which can really help on large, complex projects.

    No, it wont do anything like that.

    The compiler will come with a simple IDE, some 'templates' for use with popular editors etc (ie: there's a command line version), but that's it.

  15. Would it be possible to obtain the AST generated by the BMX2-Compiler?

    I've been working with Java for the past three months, and I've really grown to like the tools that Eclipse (or the Springsource Tool Suite, in my case) gives you. Code refactoring, smart autocompletion, grammar checking while writing the code really speed up the development process, not to mention the crazy things you can do with automatically generated code through frameworks like Roo. It's obvious that you can't implement even a fraction of these features yourself, but if one could obtain the AST of a piece of BMX2-Code, it would make it a lot easier for others to do that. Having a good set of tools made by the community would itself make BMX2 more attractive.
    If you would even go as far as making the AST-generator open source (as unlikely as that is), new features could be implemented into BMX2 by the community. Or at least being able to add custom AST transformations (similar to Groovy) would be awesome.

  16. cant wait to get my hands on this!!

  17. Wow. I just came by here via the blitz forums having sensed a disturbance in the force. Glad I did as this is now my #1 most wanted piece of software. Happy to help in any way I can. Languages I use: HTML5, JavaScript, ActionScript, Python, Objective-C. Good luck! matt

  18. Also, as an 80s child and previous Blitz user, I think simply calling it BMX would be cool.

  19. OK I got my andriod phone.. now I need a development tool.. come on mark - lets all channel our creative energies towards NZ.......

  20. How about Redline as a product name? Indicate's speed, and like Matt above, as a 80's child, Redline was a brand of BMX (bicycle) that 80's kids aspired too, so I sort of kooky link to the BMX moniker there......


  21. Mark should do a competition - come up with a winning name and get a free copy of bmx2 & possibly a free trip to NZ living in marks shed.

  22. I am extremely excited to see some support for generics. That was probably the thing that I most missed in BlitzMax.

    Do you think we'll see support for interfaces? (IE, implement multiple interfaces, inherit from only one base.)


Note: Only a member of this blog may post a comment.