Redline Smalltalk

Emitting JVM Bytecodes

2012-05-01T00:00:00-07:00

Introduction

Redline Smalltalk supports emitting JVM bytecodes in the body of your Smalltalk methods and blocks, simply use the pseudo variable JVM as the receiver and send it messages. This post will hopefully provide enough detail for you to use the bytecode facilities of Redline Smalltalk in your own classes, enabling you to call upon any feature of the Java Virtual Machine at the lowest possible level.

Why

Full interoperability with the Java Virtual Machine (JVM) requires being able to talk to it at the lowest possible level, and that level is JVM bytecode which is executed by the JVM. At this level all the functionality of the machine is available to you, including the ability to integrate with any classes running in the JVM regardless of the language they were created in. Redline Smalltalk requires the ability to load any class, call any method of a class, access any field of a class, manipulate the JVM stack and to query methods arguments and get and set local variables.

How

To enable bytecode manipulation while also keeping a Smalltalk feel in the syntax the pseudo variable JVM was introduced. This pseudo variable can also be thought of as macro because it is evaluated at read time (not at runtime) to allow bytecodes to be emitted into the stream of bytecodes to be executed before execution begins. The pseudo variable JVM is a pseudo variable like ‘true’, ‘super’ or ‘self’ allowing it to be used as a receiver in message expressions or cascaded message expressions to describe the bytecodes to be emitted. During read time the receiver in a message or cascaded message expression is checked and if it is found to be ‘JVM’ then a special analyser is used to process the remaining elements of the expression. This analyser will emit bytecodes into the stream according to the expression being analysed. For example:

  JVM invokeVirtual: 'st/redline/PrimObject' method: 'javaValue' matching: '(Ljava/lang/Object;)Lst/redline/PrimObject;'.

The above example will emit the bytecode to invoke the virtual method ‘javaValue’ with the signature ‘(Ljava/lang/Object;)Lst/redline/PrimObject;’ on the object on the top of the stack which is assumed to be of the type ‘st/redline/PrimObject’. The result of executing these bytecodes will be a value on top of the stack. Multiple JVM expressions are typically written as a cascaded message expression, which is just a stylistic choice which I find more clear. For example:

  JVM aload: 1;
      invokeVirtual: 'st/redline/PrimObject' method: 'javaValue' matching: '()Ljava/lang/Object;';
      checkcast: 'st/redline/stout/RouterRegistry';
      aload: 1.

Note that while you are emitting bytecode Redline still decorates these with source file names and line numbers so you can step through the bytecode with a debugger.

Usage

The list of messages you can send to the pseudo variable JVM is listed in table #1 with the detail of the arguments listed in table #2. However this is subject to change as this is documentation and code and documentation are not always in sync. When in doubt check the source file JVMAnalyser.java in the Redline Smalltalk distribution.

Table #1 – Bytecode selectors for pseudo variable JVM.

Selector	Description
getStatic: classFqn named: fieldName as: returnDescriptor	Get from the class ‘classFqn’ the static field named ‘fieldName’ as type ‘returnDescriptor’.
ldc: literal	Load a constant onto the stack.
aload: index	Load the local variable at ‘index’ onto the stack.
invokeVirtual: classFqn method: methodName matching: parameterDescriptor	Invoke the method ‘methodName’ of owning class ‘classFqn’ that matches the signature ‘parameterDescriptor’. Note that the method ‘methodName’ may be overridden in the receiver and the proper instance should be invoked.
invokeSpecial: classFqn method: methodName matching: parameterDescriptor	Invoke the method ‘methodName’ of owning class ‘classFqn’ that matches the signature ‘parameterDescriptor’. Note that the method ‘methodName’ must be an initializer, private method in the receiver or its superclasses.
invokeInterface: classFqn method: methodName matching: parameterDescriptor	Invoke the method ‘methodName’ of owning interface ‘classFqn’ that matches the signature ‘parameterDescriptor’.
new: classFqn	Create a new instance of the class ‘classFqn’ and put a reference to it on the stack.
checkcast: classFqn	Check the reference on the top of the stack is an class that can be cast to a ‘classFqn’.
arg: index	Push the argument at ‘index’ onto the stack. This isn’t a bytecode but a helper we have added.
<opcode>	Any bytecode opcode that does not take an argument. See table #3

Table #2 – Identifiers

Identifier	Description
classFqn	A class fully qualified name, which is java is a package name with all ‘.’ replaced with ‘/’. For example: st.redline.PrimObject has a classFqn of st/redline/PrimObject.
fieldName	The name of the field within the Class. No special decoration or formatting.
methodName	The name of the method within the Class. No special decoration or formatting.
returnDescriptor	The type of the return value or argument. This is typically in the form L<classFqn>, For more information see Method Descriptors http://docs.oracle.com/javase/specs/jvms/se5.0/html/ClassFile.doc.html#7035
parameterDescriptor	The types of the arguments passed to a method. For more information see Method Descriptors http://docs.oracle.com/javase/specs/jvms/se5.0/html/ClassFile.doc.html#7035
literal	A literal is a non null Integer, a Float, a Long, a Double a String.

Table #3 – Opcodes that don’t take an argument.

NOP, ACONST_NULL, ICONST_M1, ICONST_0, ICONST_1, ICONST_2, ICONST_3, ICONST_4, ICONST_5, LCONST_0, LCONST_1, FCONST_0, FCONST_1, FCONST_2, DCONST_0, DCONST_1, IALOAD, LALOAD, FALOAD, DALOAD, AALOAD, BALOAD, CALOAD, SALOAD, IASTORE, LASTORE, FASTORE, DASTORE, AASTORE, BASTORE, CASTORE, SASTORE, POP, POP2, DUP, DUP_X1, DUP_X2, DUP2, DUP2_X1, DUP2_X2, SWAP, IADD, LADD, FADD, DADD, ISUB, LSUB, FSUB, DSUB, IMUL, LMUL, FMUL, DMUL, IDIV, LDIV, FDIV, DDIV, IREM, LREM, FREM, DREM, INEG, LNEG, FNEG, DNEG, ISHL, LSHL, ISHR, LSHR, IUSHR, LUSHR, IAND, LAND, IOR, LOR, IXOR, LXOR, I2L, I2F, I2D, L2I, L2F, L2D, F2I, F2L, F2D, D2I, D2L, D2F, I2B, I2C, I2S, LCMP, FCMPL, FCMPG, DCMPL, DCMPG, IRETURN, LRETURN, FRETURN, DRETURN, ARETURN, RETURN, ARRAYLENGTH, ATHROW, MONITORENTER, or MONITOREXIT.

Cheating

Remembering and working with bytecode can be daunting however there are tools you can use to make this much easier. Under the covers Redline Smalltalk uses the ASM library to manipulate bytecodes. Along with this fantastic library is a tool for Eclipse that allows you to view Java source as the set of ASM calls to output the equivelent bytecodes. By writing the code you want in Java and viewing it with ASM you can see the sequence of statements you will need. See picture #1 for an example. Translating from this ASM output to Smalltalk JVM messages is not too difficult.

Conclusion

Providing as low a level access to the JVM as possible means you will have the full power of the Machine at your disposal. There should be nothing you want to do in your Smalltalk that the JVM is capable of that you cannot do.

Adopt a Class

2011-12-22T00:00:00-08:00

Redline Smalltalk requires a runtime of Smalltalk classes that people can work with to build their own Applications. There are quite a few Smalltalk classes in a simple default runtime, a few too many for the small set of core contributors to do on a short time frame, and we need your help to get a Smalltalk runtime done. What we are asking is that you adopt a Smalltalk Class and implement it with tests. The adoption process is very simple, and class class, we at Redline and the Community will love you for it.

To adopt a class you should check out the Redline Smalltalk source code and add your name to the class you would like to adopt in the file ‘OBJECTS-AND-PROTOCOLS’ and then commit that change and issue a pull request. When we accept the pull request the class is yours. Any conflicts will be resolved on a first pull-request basis.
Update: We have tried to make adopting a class easier, please see “Executing Smalltalk Tests” in the Getting Started with Redline Smalltalk post.

The file ‘OBJECTS-AND-PROTOCOLS’ lists the Smalltalk classes we are implementing for the first version of Redline Smalltalk. These classes are defined in the Blue Book (Smalltalk-80 The Language and its Implementation) along with their associated protocols.

A requirements of adopting a class is that you implement a test for each method of the class in an associated Test.st file. For example, if you adopt Object then you will have an ObjectTests.st file containing tests for each method on Object. Should you find a bug during implementation or testing then please raise an issue in the main github issues list. During implementation you may find you need a new primitive in which case you are free to implement it or you can raise an issue to have it implemented by James or Rob (experts at this now).

While the runtime is being implemented we have a very minimal testing framework to assert truths about your methods. See the file Assert.st for the provided methods. This form of testing should be sufficient to get us started and provide some level of confidence things work correctly, while protecting us from possible regression errors. We will have an SUnit or similar framework soon.

It is recommended that you look at the Getting Started with Redline Smalltalk post before adopting a class.

Smalltalk on the JVM

2011-12-21T00:00:00-08:00

James’ ESUG presentation from this August has been posted. Get some popcorn, sit back and enjoy a magical ride through the trial and tribulations of implementating a Smalltalk on the Java Virtual Machine.

Make your movie watching experience even more fun by trying to sync up the video and the slides I posted back in September.

The Road to Intelli-J

2011-12-20T00:00:00-08:00

So this is the skinny:

At Smalltalk Solutions earlier this year James and I announced that Jet Brains had agreed in principle to add support for Redline Smalltalk to Intelli-J. Well the time is here, we’ve met the basic requirements originally discussed with Jet Brains early in 2011 for Redline Smalltalk support to make it onto their roadmap.

We need your help to get it moved up the priority chain. Jet Brains has a voting system where registered users vote up features they want to see implemented. As of today, there is now one for Redline support. The next step is pretty simple…

We need everyone to sign up for a Jet Brains account ( or login with your Google or Yahoo account ) and vote for Redline support.

We need you to tweet about it.
… to email people about it.
… to google + about it.
… to stand on the street corner talking about how the world is going to come to an end unless this happens.

This could end up being a big deal for us and your help is greatly appreciated.

Update:

This post is currently on the front page of Hacker News and it has been pointed out, that lacking in context, it could be confusing as to why someone into Smalltalk would care about Intelli-J support. Redline Smalltalk is an implementation of Smalltalk that runs on the JVM, designed to work within the JVM ecosystem. For us, that means that it should work with the tooling that JVM developers are familiar with: Eclipse, Intelli-J et al. There is more information elsewhere on the site, but this site is still a work in progress. If you are interested in learning more, right now your best methods are the mailing list and our brand new IRC channel.

Update Redux:

Dmitry Jemerov posted the following to the Intelli-J issue for Redline Smalltalk support:

As the development lead of IntelliJ IDEA, I would like to clarify the situation and to make expectations a bit more reasonable. The RedLine blog at http://www.redline.st/blog/2011/12/20/road-to-intellij.html says:

“James and I announced that Jet Brains had agreed in principle to add support for Redline Smalltalk to Intelli-J”
This was not a decision made by JetBrains management. We do not see Smalltalk support bringing any significant business value for the company, and therefore we do not currently have any plans to invest any significant development resources into Smalltalk support, or to bundle Smalltalk support with a future version of IntelliJ IDEA.

What we can actually do is provide some basic foundation for the plugin, using the parser generation tools that we’ve recently developed, and provide assistance to the community in developing the plugin further.

Unfortunately that decision is very unlikely to be affected by the number of votes for this issue.

We want to thank everyone who has voted up this issue so far and ask you to continue doing so in order to demonstrate the support. However, we feel it is important to for everyone to understand the situation as it stands and what your support can and can not accomplish.

The level of support we have received so far as really inspired us and hope to see plenty of you on the mailing list and IRC channel in the near future.

IRC here we come

2011-12-19T00:00:00-08:00

Yesterday afternoon “Steven! Ragnarök” asked on the mailing list if anyone was interested in setting up an IRC channel for Redline. A few minutes and some ChanServ manipulation later and I had registered #redline-st for our usage.

We’re a small community right now so it isn’t the most active place but, stop on by so you can say “I was there when…”.

Hope to see you there!

Server: irc.freenode.net
Port: 6667
Channel: #redline-st

Why Smalltalk? (Slight Return)

2011-11-30T00:00:00-08:00

A while back, the ‘Why Smalltalk?’ question came up on the Redline Smalltalk mailing list. What follows is the question as it was asked and James’ answer to it. A couple weeks past, I posted my response and meant to post James’ but work and Redline got in the way of posting this until now.

What makes Smalltalk special enough that you strongly feel it’s better suited than the usual suspects for software development?

I think Smalltalk is more productive because there is less to remember, everything is a message to an object. This message passing is everywhere – it is the language itself. When you remember this you can do anything the language and runtime allows. Conversely languages where you need to understand the runtime in addition to language syntax require more brain power. I’d like to save that brain power for the problem at hand rather than the tooling I’m using to solve the problem.

To me, this is one of the brilliant things about lisp, almost everything is expressed in a uniform way. Is Smalltalk better than Lisp? Of course, and of course not. It depends on how you think and which problem you are trying to solve.

Why Smalltalk?

2011-11-15T00:00:00-08:00

Earlier today, the ‘Why Smalltalk?’ question came up on the Redline Smalltalk mailing list. What follows is the question as it was asked and my first stumbling attempt at answering it.

What makes Smalltalk special enough that you strongly feel it’s better suited than the usual suspects for software development?

That would be a very lengthy answer and honestly, I don’t think it would matter. Words are not really going do justice. I used Smalltalk every day for about 9 months before it suddenly hit me how much more productive I was with it than most languages I had ever used.

I could talk about doing debugger driven development where you write the tests or client code and just it the code and then proceed to add all the supporting methods from the debugger and just restart from point X and how powerful have resumable exceptions are and what amazing things ‘thisContext’ brings but unless you’ve used it and experienced it for a while, that isn’t going to mean much.

I could talk about how in many ways Smalltalk is similar to Ruby but in my opinion, a ‘saner’ ruby- I am very productive in Ruby as a language and can see being even more productive with a file based Smalltalk sans the environment because of those ‘sanity’ checks.

I could talk about how powerful the Smalltalk integrated environment is for doing development and how eventually I want to have the option for the same in Redline while maintaining being able to interoperate with a file based world.

And all that just scratches the surface.

It is however a question I need to try and find a succinct way to answer as the Redline site is going to need to answer it. I originally avoided answering that part of the question because I don’t have a good succinct answer. The is an attempt at a succinct answer and I think it sucks. But, I am working on it.

Smalltalk on the JVM

2011-09-01T00:00:00-07:00

The slides from James’ 2011 ESUG presentation ‘Smalltalk on the JVM’ are now available.

The presentation covered the challenges that we’ve had implementing a dynamic language like Smalltalk on the JVM and how we’ve gone about addressing those issues. While we certainly aren’t alone in having gone through these struggles and aren’t the only one’s to document it, we hope our experiences end up helping someone else down the road.

If you are looking to get a quick overview of how Redline works internally, this presentation could be an excellent resource.

Update:

ESUG just uploaded video of the presentation.

The Journey So Far...

2011-04-01T00:00:00-07:00

Our presentation from Smalltalk Solutions 2011 is now available. It’s an excellent primer on what we want to accomplish with Redline. Plus, as an added bonus, you get to see us dressed up as Jedi Knights. For the more adventurous amongst you, try your hand at maching the slides to video.

Enjoy.

Independent Misinterpretations 7

2010-10-29T00:00:00-07:00

Check out my appearance on episode 7 of Independent Misinterpretations – a Smalltalk and dynamic language oriented podcast with James Robertson, Michael Lucas-Smith, and David Buck.

A Fan Letter to Redline Smalltalk...

2010-10-27T00:00:00-07:00

It seemed appropriate that the first post to the Redline Smalltalk Blog was about Redline and what is in it for you. So I took a hypothetical “fan letter” to Redline Smalltalk from my other Blog and put it here (see below). Work continues on Redline with the parser complete and bytecode generation in full swing. The Smalltalk runtime is being ported from the Pharo Smalltalk sources and development continues on the Eclipse Plugin.

I was inspired by this post by Kent Beck to write a fan letter to Redline Smalltalk, written in the voice of a deliriously satisfied user. This is an excellent way of articulating a vision.

Dear Redline Smalltalk,

I have been using Redline Smalltalk at work and in personal projects for a while now and I just had to let you know what a joy it is to use and how productive I am with it.

My work situation means I have to work on the Java Virtual Machine and I have been looking at alternatives to Java so I could be as productive as possible and stay away from those little Java annoyances and hurdles that slow me down. Having a background in Smalltalk I jumped at the chance to put your Redline Smalltalk through its paces and I have to say I couldn’t tell the difference between your Smalltalk and the others, in fact I think yours was a little faster.

At home I like to use IntelliJ and your plug-in support in the IDE is brilliant, the mix of old style Smalltalk-80 browser, refactoring and file based support is smooth and complete and just what I needed, especially the source level debugging and profiling which I could not live without. At work we use Eclipse and I can’t tell the difference between the plug-ins so now I’m getting a few other developers to try it, and I think I made some converts.

One developer at work, Steve, keeps banging on about how Ruby On Rails is this and how Ruby On Rails is that, how we need rspec, gem, rake, and how awesome it is to script things and try stuff out with irb. You should have seen his jaw drop when I showed him how I can do all that in Redline and how quickly I could create, test and deploy my Seaside Web App and then push changes to different environments with a simple commit, even while they were running. You really did think of everything to make things complete and integrated with Smalltalk everywhere.

I wondered what you could add in the next release and I just can’t think of anything but if I do, Ill be sure to let you know.

Thanks for a wonderful product that has really changed the way I work and made it fun again.

Regards,

John Q. Geek.