Monkey2 translator output.

One of the things I was hoping to achieve this time around with monkey2 was 'clean' translator output. Anyone who's looked at the native code monkey1 generates will agree it's a bit of a mess!

I was hoping that the c++ output, at least, could look like a bit like 'real' c++, eg: this...

namespace mojo

class Image
   class Frame
   end
end

...could translate to...

namespace mojo{
   class Image{
      class Frame{
      };
   };
}

Alas, it's not that simple.

The main issue with nicely scoped c++ is that it turns out you can't 'forward reference' inner classes - at all! This is a new one on me, but one of the things I really like about monkey1 is how all the forward referencing issues you have in c/c++ just don't exist, and I want the same to be true of monkey2. So I have decided to abandon nicely scoped c++ output for now.

I was also hoping I could rely on c++'s function overloading to allow me to use the same function names for overloaded functions.

This mostly works, except when using overloaded member pointers. You can 'cast your way' out of the problem (see: http://stackoverflow.com/questions/4364599/c-overloaded-method-pointer) but that's pretty nasty. It's easier for monkey to just mung a new name for overloaded functions, so I've decided to go with that for now too.

So c++ output wont be as sexy as I'd hoped, but there is an upside to this: the translator is now much more 'universal' and will be easier to adapt to other target languages. This is probably what I should have aimed for in the first place.

The output WILL be cleaner than monkey1's though, mainly because, for now at least, I only have to write one translator! Having to write 5 at once was a killer, and the quality of each suffered.

...and Mojo2!

And in further news...mojo2 is on the way!

I've been working on this for several months now, in collaboration with Tony from Playniax (who has news of his own!) and it looks like mojo2 should be ready for release within a month or so.

Mojo2 is, like mojo1, fundamentally a 2D engine. However, it adds a few useful 3d features such as lighting (with normal/specular maps) and the ability to use 3d projection/view matrices. Mojo2 is ONLY compatible with targets that support the OpenGL module. That is, the desktop, ios, android and html5 targets.

A brief overview of features:

You can render directly to the app window or an image using the Canvas class, eg:

Local canvas1:=New Canvas                  'render to app window
canvas1.SetViewport...
canvas1.SetProjection2d 0,VWIDTH,0,VHEIGHT 'set 'virtual resolution'.
canvas1.Clear
canvas1.DrawRect...
canvas1.DrawImage...

And to render to image:

Local image:=New Image( 256,256 )
Local canvas:=New Canvas( image )       'render to image!

The canvas methods available should be familiar to anyone who has used mojo1, although a  few have been tweaked/renamed here and there.

You can also perform simple 'direct mode' lighting using canvas, eg:

canvas.SetLightType 0,1                'turn on light 0.
canvas.SetLightRange 0,100             'set light 0 range.
canvas.SetLightPosition 0,320,240,-25  'set light 0 position.
canvas.DrawImage...                    'image will be lit!
canvas.SetLightType 0,0,               'light 0 off again.

You can use up to 4 lights in 'direct mode' like this. Shadows are also possible using direct mode, but it starts getting tricky...

To help deal with multiple lights and shadows, mojo2 also provides a 'Renderer' class that provides a sort of 'virtual scene manger'. You add layers and lights to a renderer, and when you call Render() it will take care of all the tricky multi-pass rendering stuff. It is NOT a complete scene graph, but something you can plug a scene graph (simple or complicated) into.

There is also simple 'shader support' in that you can write glsl shader code that samples textures and produces values suitable for lighting.

There are still a few details to be worked out, but that's about the jist of it! Price wise, it'll be added to the commercial monkey-x package and will be free to existing monkey-x owners.

Bye!
Mark

Monkey2!

Ok, some fairly big news: I've decided to start work on Monkey2!

I'm going with an open source/crowd funded approach this time, as I don't consider the idea of selling languages to be commercially viable these days. I am planning to start with a patreon project (see: https://www.patreon.com/) which I hope to have up in a week or so.

Is there any point to writing a new language? In a way, probably not, as I'm constantly reminded each time I read a slashdot thread about new language XYZ! But I do think there's a core bunch of people who like using BRL languages (myself included) and since I also really like making them, I'm gonna try to figure out a way to continue to do just that as a full time thing.

The basic idea is to fix the problems in the monkey1 language, add some cool new language features, and take the build/runtime system in a much more blitzmax-ish direction. To this end, monkey2 will initially only support c++ targets that have a posix-ish api, which effectively means that only the desktop, ios and android targets will be supported. However, since the overall approach is basically the same as monkey1 - ie: everything ends up going through a translator - the potential is there to support more targets in the future. Alas, as I found out with monkey1, emitting code in bunch of target languages is the 'easy' bit...

Losing html5 hurts, but without it the range of 'lowest common denominator' features becomes much richer, and the job of supporting all those features much simpler. Most of the other targets are pretty much dead now, so I'm not too sad about losing them. Besides, they'll live on in monkey1 which I fully intend to continue to support and tweak.

Why not just 'fix' Monkey1? Mainly because it's gotten pretty tricky to sanely tinker with in certain areas - areas that I really, really want to tinker with. It was my first attempt at an OO language with generics, and I will freely admit that I didn't quite nail it first time around. But I've been hacking away on Monkey2 for the last week or so, starting from the ground up with a new parser, a new approach to semantic analysis etc, and I am very happy with how it's going and (finally) feel confident I can do much better this time.

All in all, it is my hope that monkey2 will find a nice balance between monkey1's sexier language and blitzmax's 'hackability'.

There is nothing to release yet, but I've added a Monkey2 forum to monkey-x.com so feel free to post comments and questions there.

Bye!
Mark

BlitzPlus source code now available on Github!

Hi,

The BlitzPlus source code is now available on github:

https://github.com/blitz-research/blitzplus

It's a bit messy (there's probably lots of stuff that can be trimmed etc), and you'll need MSVC 6.0 to compile it 'as is', but it's all there!

I'd also like to reassure Monkey users that when/if I one day 'retire' Monkey (ie: stop selling it), I will also be releasing the full source code to the currently proprietary bits of Mojo in a similar way. Ditto Blitz3D/BlitzMax...

Peace out!
Mark

Monkey stuff!

Ok, this was initially going to be a forum post, but it got a bit longwinded so it's probably worthy of a post in my poor, neglected blog.

On the Monkey front, I've pretty much got async loading of images/sounds going now - usage looks something like this:

Class MyApp Extends App Implements IOnLoadImageComplete

Field _image:Image

Method OnCreate()
Image.DefaultFlags=Image.MidHandle
LoadImageAsync "cheerio.jpg",,,Self
SetUpdateRate 60
End

Method OnUpdate()
UpdateAsyncEvents 'must do this!
End

Method OnRender()
Cls
If _image DrawImage _image,MouseX,MouseY
End

Method OnLoadImageComplete:Void( image:Image,path:String,source:IAsyncEventSource )
If image
'image loaded OK!
_image=image
Else
'problem loading image!
Error ""
Endif
End
End

This works pretty much identically on all targets, except for PSM which can't do async image loading, even with a thread - it complains when you try to use the graphics context from a background thread. In this case, it's done synchronously so still at least works.

On c++/java targets, the load executes on a separate thread. On html5/flash, 'onloaded' callbacks are used internally to 'fake' threading. Note that on targets with embedded data (xna/flash) this is really just decompressing data on a separate 'thread', so may not be all that useful. Also, on OpenGL targets textures must be created on the main thread so there may still be a 'pause' after the image has loaded in the background when it gets sent to GL. This is theoretically fixable, although my experience with multi-thread GL contexts in bmx was less than encouraging...

In addition, image/sound loaders can now load stuff outside the data directory where possible. File paths are now more URL-ish, so (on some targets) you can do things like:

Local image:=LoadImage( "http://www.monkeycoder.co.nz/tmp/cheerio.jpg" )

This is down to the capabilities of the underlying native loaders, so for example http: currently only works on html5/android/ios. On Flash, it works if the image comes from the same domain, otherwise you need to configure the resource server - something Monkey can't help with. In the case of c++ targets, this can be used to load stuff from the filesystem but not http: (yet).

This has meant some tweaks to the file path system, as the path "cheerio.jpg" can now mean several things: It could be a file in the app's 'data/' dir, or it could be a file in the (c++/java) current dir or the (html/flash) document base.

To deal with this, I've added the 'pseudo' URI monkey: which works a bit like http: and can be used to locate monkey specific resources. Currently, there's only monkey://data/ to get at resources in the app's data dir.

To maintain compatiblity, Mojo will for now convert all relative paths into monkey://data/ paths. It will not convert paths that start with "./", "/" or "blah:", so to load something from the 'real' current dir you will need to use the "./" prefix.

This means these are equivalent:

Local image:=LoadImage( "cheerio.jpg" )
Local image:=LoadImage( "monkey://data/cheerio.jpg" )

But these are not:

Local image:=LoadImage( "cheerio.jpg" ) 'gets converted to "monkey://data/cheerio.jpg"
Local image:=LoadImage( "./cheerio.jpg" ) 'does not get converted

Whew! Sounds complex, but I think it's a reasonably sensible approach.

The new Mojo async loaders are built on a new interal async event system in the brl module, brl.asyncevent. The core of this system is the IAsyncEventSource interface that provides a single method...

Interface IAsyncEventSource
Method UpdateAsyncEvents:Void()
End

...and a bunch of global functions...

Function UpdateAsyncEvents:Void()
Function AddAsyncEventSource:Void( source:IAsyncEventSource )
Function RemoveAsyncEventSource:Void( source:IAsyncEventSource )

UpdateAsyncEvents() needs to be called regularly - once per OnUpdate() should be enough - and is what actually updates/polls the state of any async operations currently in progress. This is when various OnBlahComplete methods will be called.

You can add your own async event sources by writing a class that implements IAsyncEventSource and using AddAsyncEventSource/RemoveAsyncEventSource to add/remove objects of this class to the global list of async event sources.

Note that there is no 'event' class as such - each async event source must provide it's own mechanism for notifying the main app of progress/completion, eg: by providing an IOnBlahComplete interface.

There's also a new AsyncStream class that uses the async event system to let you read/write from some types of stream in the background. Currently, only async TCP streams are supported, but future versions could include include AsyncFileStream (once we HAVE a file stream!) for streaming from 'slow' filesystems such as Dvd, and even AsyncAudioStream for realtime mixing.

I've also made several reasonably major changes to the config var system.

First up, you can now use '+=' to append to config vars (you still can't overwrite them with '=' though). This will append the RHS to the var, inserting a ';' if necessary to keep 'em separated. This is very much a WIP feature right now and mainly for the sake of REFLECTION_FILTER, although MODPATH and data file filters might also work. Also, both ';' and '|' are now valid separators for REFLECTION_FILTER.

The reflection module no longer does a #REFLECTION_FILTER="*" by default, so it's up to you to set the correct filter(s) or it will be empty. Being able to append to config vars now means modules can add themselves to the filter with eg: #REFLECTION_FILTER+="mymodule*|mojo*" etc.

CONFIG.TXT files are now plain Monkey files, although they're only ever preprocessed. I've renamed them CONGIG.MONKEY to try and limit any confusion here. Note that this means each line of CONFIG.MONKEY must now start with a '#', just like real Monkey code.

The handling of bool true/false config vars has been changed/unified - you should now use the Monkey tokens True or False (no quotes, case insensitive - ie: Monkey code) instead of "true" or "false" for all bool vars. There's a minor issue with bool->string conversion here, since all config vars are actually stored as strings and Monkey doesn't support bool->string conversion. To deal with this, I've hacked the preprocessor so true gets converted to "1" and false gets converted to "0". This will only be an issue for those writing code that uses config vars internally, but it's something to be aware of. It's meant that I had to change any existing code that compared config vars with "true" to compare with "1" instead.

This all represents some potentially 'breaking' changes for anyone who's mainting their own version of trans - apologies for that, and perhaps I should have saved these for a future release as there are more breaking changes coming...

As for the future of Monkey, here's the current plan...

A Windows 8 C++/DirectX target is underway and will hopefully be done 'soon'. I have gone with C++/DirectX because it seems to be the lowest level/highest performance/most flexible way to go. C# appears to involve using some GUI markup language called XAML, and doesn't appear to be able to access directx, erm, directly. Please correct me if I'm wrong here, but the impression I get is that MS is going 'native first' to compete with ios, android.

Trans-wise, I want to add the ability to import source/lib files from modules into a project. Just how to do this cleanly became clear once += was added, eg: SRCS+="{CD}/native/lang.${LANG}". This way, the 'lists' of SRCS, LIBS etc are completely target dependant and 'quoted imports' can eventually be killed off, something I've always wanted to do.

Also, I'd like to make it easier/possible to 'roll your own' targets. This is the biggy, and I think there are 3 main changes that need to be made to make this possible:

First, targets should have their own modules/ dir, allowing them to provide target specific monkey APIs without having to provide a separate modules/ package.

Second, all the 'native' app-specific classes/code such as view, viewcontroller, activity etc should be moved from Mojo to the target. This at first sounds like a step backwards, but it means each target becomes entirely self contained so can easily be kludged as necessary to support ad systems, social networking etc. It also means you can use interface builder and other high-level target SDK systems to create the initial target environment etc. So basically, target projects will become standalone, runnable projects that can be built and run without ANY Monkey code (except perhaps a nop Main) and the target modules provide the Monkey side interface to the target.

Finally, the 'Target' class in trans should become 'Builder', and targets should specify which builder to use in a TARGET.MONKEY file. Examples of builders would include GccBuilder, XcodeBuilder, MsvcBuilder, MonoBuilder, FlexBuilder etc. So, for example, both the Mac ios and glfw targets could use the XcodeBuilder.

The goal here is for people to be able to simply copy and paste a target dir, tweak the source code and/or  TARGET.MONKEY settings, and end up with a standalone target that can be dropped into any Monkey release.

The last 'big job' regarding trans is to split up the monolithic output 'main.cpp' style files into per module sources, and to be able to compile them separately. Once all of the above is going, this should be quite doable.

Last but not least, there WILL be an official Linux version, although it will be Makefile based in keeping with the Linux spirit! Given what MS and Apple are doing with forcing developers to sign apps etc, the day when Linux becomes the last, free OS is, I fear, coming much sooner than I once thought.

As for Steam-on-Linux, I'm in two minds about this. On the one hand, I agree with Linus's POV that anyone should be able to do anything with Linux, even DRM etc. On the other hand, there is zero evidence to suggest that Valve/Steam, as cool as they are now, wont eventually end up abusing their potential power. I already don't like the 'exclusivity' aspect of having to get your game approved for Steam, whether or not that's by popular vote or whatever. I think overall I tend to agree with Linus here: freedom has to be *absolute* or it's not freedom - even the freedom to be an asshole. But I also think we also need guys like RMS to keep reminding us where we stand in terms of the big picture, or we'll start forgetting that THERE IS ALWAYS AN ALTERNATIVE!

Peace out!
Mark

Surprise!

Surprise!

Well, it's been quite a while since Monkey was officially released, and I finally feel like things are settling down a bit!

So, a few thoughts about Monkey...

As with any product release, there were a number of unforseen issues to be dealt with, but perhaps the scariest was the 'crappy graphics performance on Android' problems the arose immedately after release.

I made several goofs here, but the major one was probably underestimating how grunty typical Android hardware actually is. I initally went with the 'Canvas' API under the impression it would work anywhere, and benefit from OpenGL acceleration where possible. Well, it turned out I was right about the former, but way off about the later - the Canvas API is SLOW and does not appear to be in any way accelerated.

The situation improved by moving to OpenGL1.0, but Monkey games still weren't cutting it compared with other games out there. It wasn't until I moved to GL1.1 + VBOs ('vertex buffer objects') + weird-driver-behavior-workarounds that we really started getting acceptable speed.

This really took longer than it should I have. I should have tried to work out what sort of hardware/OpenGL drivers were actually in common use, and gone from there. Instead, I kind of got stuck trying to work out what a bunch existing (but dated) sample code was doing - perhaps a case of 'too much' information in way? Still, I learned a lot in the process I guess.

Another thing that bit me on release was the 'generics' system used by Monkey. I initially modelled Monkey generics on the system used by Java. However, for various reasons (the main one being my own inexperience at implementing generics in a compiler) this turned out to be a BAD choice, and I've only just managed to get generics into what I'd call a really usable state. And it turns out the way I've ended up doing generics is MUCH simpler and the potential for improving it much greater now.

I actually feel that in an odd way that Monkey is the first compiler I've written that I really have a good handle on. Previous compilers have sort of reached both the 'finished' and 'unmanageable' started at about roughly the same time - more or less by design! But with generics now cleaned up, I feel very comfortable with the Monkey code. It's not perfect by any means, but I still have a rough idea of how everything works, where the weak bits are etc.

And what next for Monkey? My current list of 'big picture' improvements under consideration includes:

* Better control over app data - in particular, what data is used on what targets, and how. Currently, any data considered usable by a target is added to the target project based purely on file extension. But this can mean some data ends up getting duplicated in some situaitons, esp. when you are required to provide the same data in multiple formats to handle target limitations. The current plan here is for some kind of config/control file in the data dir.

* Reflection - ie: the ability for your app to be able to dynamically call itself. And I don't think this will actually be all that hard. One thing that has struck me since release is just how flexible a translator is - you can actually do much more with a translator than I realized. To provide reflection data for an app should just involve generating suitable reflection classes/code - in Monkey - for each app decl, and then handing the whole lot over to the translator. The native target translators shouldn't have to do any thing.

* Interpreter - which would be much more useful/intersting if built on top of reflection. For example, if you want interpeted code to be able to call DrawImage, it'd be preferable for the interepter to just be able to use reflection. You'd hopefully then be able to get rid of the need for users to write any 'glue' code. Interpretable Monkey would also be (more) easily debuggable, but I'm not sure how useful that'd be in practice, as interpreted Monkey wont be able to do everything native Monkey can, and is likely to be slowish. Still, you'd be able to stop/start interpreted code even on callback based targets, so perhaps it'd be OK. If not...

* Stop/Start style debugger for the GLFW target. As Monkey users are well aware, debugging can currently be a bit, erm, challenging. Basically, if your app tries to access a null object or index an array beyond bounds, the app stops and you get a stack trace - and that's it. Unfortunately, a proper stop/start debugger for all targets would be impossible, as many targets cannot be stopped as they're callback driven. Therefore, I think the most practical solution to the debugger problem would be to concentrate on making a decent debugger for the one target that can really support it - GLFW. This isn't ideal, and I'm still considering options for debugging, but given the nature of Monkey, it's kind of a tricky one!

I've got lots of smaller tweaks I'd like to make too, but these are the 'biggies' I'm currently considering.

And on the non-Monkey front...

I've just finished an excellent book called 'Ready Player One' - not by any means a literally masterpiece, but an incredibly fun cyberpunky adventure story that will appeal heaps to anyone who was involved in computers/gaming in the 80's, ie: me! Very highly recommended!

I totally loved the recent True Grit movie too, somewhat to my surprise as I'm not generally a big fan of the Cohen brothers stuff. I usually find their movies highly entertaning, cleverly plotted etc, but ultimately pretty bleak. I mean, yes, there are pyschos out there with bad haircuts, doing nasty things to people with electric cattle prods...so?!? Probably related to me aversion to 'torture porn'. But I found True Grit to be very different. It's much more of a straightforward adventure story, with hugely likeable, if flawed, characters. Even the bad guys were kind of cuddly in their own way! I also checked out the book and 1969 version, and both are highly recommended too.

And if you're into Anime, I highly recommend Katanagatari. It's got a highly unique visual style, almost reminiscent of the game 'Okami', lots of clever, frequently funny dialog and plenty of cool fighting sequences involving Ninjas and the odd sexy nun! It's also nice and unpredictable, always a plus for me.

Finally, a big shoutout to anyone 'occupying' anywhere right now! Not sure what will ultimately be achieved, but I do thinks it's managed to raise the odd eyebrow in the upper/political classes, and does represent a faint hope that democracy wont lose out to complacency in the end. I'm not anti-capitalist or anti-free market and I'm definitely not anti-iPhone (apparently, protesters are allowed to use capitalist-derived technology), but I do think there's a very good case to be made that a very small bunch of wealthy, privilaged people are running off with all the loot (regardless of the havoc they wreak in the process of obtaining it) while things just get harder for everyone else - often as result of the reckless behavior of said looters! And govts, left or right, seem oddly uninterested in doing anything about it.

Websites majorly down!

Hi,

As you may have noticed, both the www.blitzbasic.com and www.monkeycoder.co.nz sites have been down for over a day now.

Apologies for this, but it looks like there's been a pretty serious server crash. Our webhosts are currently working on getting the site back up, but we don't know how long that will take.

So please bear with us, normal transmisson will resume as soon as possible.

Bye!
Mark