How much memory does my ColdFusion variable really use? – Part I

by kai on 07/03/2010



Recently I’ve been dealing quite a bit with ColdFusion from an infrastructure point of view – clustering, monitoring, system tuning etc. One particular thing I (and I know of at least some others) have found missing was an easy to use and realistic way of figuring out how much memory a particular ColdFusion variable really uses. There are a few approaches around that regularly pop up on mailing lists to roughly estimate such a figure from code. There are also a few tools such as SeeFusion, Fusion Reactor or the ColdFusion Server Monitor that provide assistance when it comes to figuring out JVM memory usages or even are able to break usage down to a scope such as the application scope.

All that’s good information, but it didn’t provide the information I wanted in this case. I literally wanted to be able to create an array or a structure in CF and see how much memory the instantiation of this variable will take. I don’t see a way to get this information from ColdFusion itself, but here’s a nice feature of the underlying Java platform to the rescue: JVM instrumentation (available from Java 5 onwards).

If you followed the link above you would have seen the getObjectSize(Object) method and its description “Returns an implementation-specific approximation of the amount of storage consumed by the specified object.”. Ah, cool. That sounds about what we want, right. Take an object – throw it into that method and get back the amount of storage used, doesn’t it?

Notice the following restrictions:

  • implementation-specific and
  • approximation

What does that mean for us? Well – basically what it comes down to is: You get an approximation, it  might not be correct to the last byte. The JVM will try hard, but it might just not be 100% exact. Also, speaking of the JVM – the number it returns will be correct for your hardware, operating system and JVM used. To give you an example – the figures I get on my 64-bit OS X 10.6.2 with JVM 1.6.0_17 might be different from someone on 32-bit Windows with a Java 5 JVM or even a JVM 1.6.0_16. So – to sum it up: YMMV.

I started playing with instrumentation a bit and then found issue 142 of Heinz Kabutz’s JavaSpecialists newsletter. In there, he provides a JVM-instrumentation based memory counter that seemed to do exactly what I wanted to do. The idea is that you have a MemoryCounterAgent class that provides two static methods: sizeOf() and deepSizeOf().

The way to use his code is rather straight forward. Compile the MemoryCounterAgent.java provided there – either with a tool of your choice or use the ANT scripts Heinz has provided. The result should be a .jar file that you’d best put into the lib folder of CF respectively Java. I’ve done all this on CF 8.0.1 / OS X 64-bit installed in a JRUN4-based J2EE installation. The folder I place the .jar file in this case is /Applications/JRun4/lib.

To hook your JVM into the instrumentation agent, you need to modify the JVM arguments. You find those usually in the jvm.config file in the bin-folder of your CF or JRun installation. The idea is to add a -javaagent argument in front of the other, existing arguments that points to the .jar file you’ve created and placed in lib before.

java.args=-javaagent:/Applications/JRun4/lib/MemoryCounterAgent.jar

Restart ColdFusion and after that you’ve got the instrumentation available for ColdFusion code. Put the Java object in your CF code (it’s static, so don’t instantiate it) and call a method of it:


	oObjectSize = CreateObject("java","eu.javaspecialists.tjsn.memory.MemoryCounterAgent");
	myText = "This is a string";
	WriteOutput(oObjectSize.sizeOf(myText));

“What’s the catch?”, you might ask. Well, there are a few. First of all – the unmodified code works for a variety of different CF variables, but it pretty much provides weird results for some others. The reason for this is that a CF variable is basically a Java variable. Strings in CF are of type java.lang.String – easy. Arrays in CF are of type coldfusion.runtime.Array and most complex CF types have such a non-trivial representation in the Java world. Those Java classes contain CF-specific implementation details and hooks into other parts of the ColdFusion infrastructure that really blow out and misshape the memory estimations the JVM instrumentation provides. There are ways around that (by using reflection on CF variables to find out how they are constructed, but that will be covered in part II of this series of blog posts).

The second issue are flyweights. For all those of you who have no idea of what that means – the flyweight pattern is a way to manage memory that is used for certain types in Java and its JVMs. Using (and including) flyweights in memory calculations can produce significantly different results from not using them. Also – that will be covered in more detail in part II or III.

Anyway – there are reasonable solutions for both issues, I believe. I’m currently building a CFMemoryCounterAgent class that provides memory tracking on CF variables as close as it gets – stay tuned, more on this here in the next few days.

{ 9 comments… read them below or add one }

Heinz Kabutz March 8, 2010 at 12:52 am

Hi there, the memory counter code usually needs a bit of tweaking to work properly on your system. You should define end-points to stop the traversal of the tree pulling in whole large chunks of CF code. I would probably do that by ignoring packages that start with the framework packages.

You’ll probably write about that in your next articles though 🙂

Heinz

Reply

kai March 8, 2010 at 5:45 am

@Heinz: yes, that’s exactly what I had to do. In some instances the CF variables are tied into CF framework code and a reasonable easy variable would end up being 15 MB or so. And you’re right, it will be shown in the next post 🙂

Reply

Hendrik Kramer March 8, 2010 at 3:25 am

Railo hat eine Build-in Funktion zum Auslesen des Speicherverbrauchs von Scopes und Strukturen. Siehe z.B. hier: http://groups.google.com/group/railo/browse_thread/thread/a8900e347641e5fb/fc2f853062c89e26?lnk=raot. Ist leider noch nicht so gut dokumentiert.

Reply

kai March 8, 2010 at 5:46 am

@Hendrik: Ah, interesting. Haven’t tried it with Railo, but it looks a bit as if they’re making use of a similar approach then. Is there an instrumentation agent setup in the JVM args for Railo?

Reply

Hendrik Kramer March 8, 2010 at 6:43 am

@Kai: No, there’s no similar instrumentation setup in the Java Args. Maybe somewhere deeper in the configuration setup. And it’s independent from the underlying app server used, so it must be implemented fully between Railo and the JVM.

Reply

Mark Mandel March 8, 2010 at 9:29 am

If you are looking for an established project for calculating size of objects in Java, you can try the sizeOf() project – http://sizeof.sourceforge.net/

Reply

Dave Quested March 8, 2010 at 10:36 am

Kai, this is great stuff, thanks for sharing. Looking forward to seeing how you progress with this.

Reply

Heinz Kabutz March 8, 2010 at 11:48 am

@Mark: You will still need to define the end-points, I think, otherwise you might pull in the entire CF app every time. BTW, have a look at the README.txt file 🙂

Reply

Gabriel March 9, 2010 at 7:35 am

ha!

I won’t spoil it but the readme did make me smile. 🙂

– Gabriel

Reply

Cancel reply

Leave a Comment

{ 2 trackbacks }

Previous post:

Next post: