Skip to main content

Executing in Memory a JSP DCP from Database

This is follow-up of my previous post Executing a JSP DCP Stored in the Database, where I was presenting a proof-of-concept on executing a string representing a JSP DCP stored in the Content Delivery Database.

I wasn't too happy with the previous solution, because it was writing the compiled .class to the file system and there was no caching at all (the .class would be recompiled with every request). I can do better than that! :-)

So the new version is compiling everything into a memory byte array, then executes the in-memory class. The compiled class is placed into an LRU cache, in order to optimize performance.

The entire example is available on Google code project; below are just the highlights:

Custom JSP Tag

This is the calling code that initiates the execution of the DCP.

public void doTag() throws JspException {
    try {
        DcpClassLoader loader = new DcpClassLoader(componentUri, componentTemplateUri);
        Executable dcpExecutable = loader.getExecutable();
        String result = dcpExecutable.execute();

        JspContext context = getJspContext();
        JspWriter out = context.getOut();
        out.write(result);
    } catch (Exception e) {
        throw new JspException(e);
    }
}

DcpClassLoader

The class extends ClassLoader and is resposible for finding dynamic in-memory class for each DCP. It uses an LRU cache to store the recently created dynamic DCP classes. It also takes into account the lastPublicationDate of the DCP, which is used to invalidate the cache.

This class also instantiates the retrieved class (an implementation of the Executable interface).

If the DCP class is not found in the cache, this is the entry point into the in-memory compiler (method createExecutorClass).

public DcpClassLoader(String componentUri, String componentTemplateUri) throws ParseException {
    super(DcpClassLoader.class.getClassLoader());

    cp = getComponentPresentation(componentUri, componentTemplateUri);
    lastModified = getLastPublicationDate(cp);
    className = String.format("Dcp_%d_%d_%d", cp.getPublicationId(), cp.getComponentId(),
            cp.getComponentTemplateId());
}

@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
    ClassLoaderCache cache = ClassLoaderCache.getInstance();
    CacheEntry cacheEntry = cache.get(name);
    if (cacheEntry != null && cacheEntry.getLastModified() > lastModified) {
        return cacheEntry.getClazz();
    }

    Class<?> clazz = createExecutorClass(name);
    cache.put(name, clazz);

    return clazz;
}

public Executable getExecutable() throws ClassNotFoundException {
    try {
        Class<?> clazz = loadClass(className);
        return (Executable) clazz.newInstance();
    } catch (Exception e) {
        log.error("Exception occurred", e);
    }

    return null;
}

private Class<?> createExecutorClass(String name) throws ClassFormatError {
    DcpExecutor executor = new DcpExecutor(name, cp.getContent());
    byte[] classBytes = executor.compile();

    return defineClass(name, classBytes, 0, classBytes.length);
}

DcpExecutor

This class puts the DCP content into a Java class context (the sourceSkeleton), compiles it in memory, and returns the byte code of the compiled class.

The sourceSkeleton is in fact implementing the Executable interface (which only defines a public String execute() method). In order to compile the dynamic Java source, the resource mitza.dynamic.compile.Executable needs to be made available to the compiler (in the classpath). Therefore, the compiler accepts an Iterable<Class> containing the classPath elements.

private static final String sourceSkeleton =
        "import java.io.ByteArrayOutputStream;\r\n" +
        "import java.io.PrintStream;\r\n" +
        "import mitza.dynamic.compile.Executable;\r\n" +

        "public class %s implements Executable {\r\n" +

        "    public String execute() {\r\n" +
        "        ByteArrayOutputStream output = new ByteArrayOutputStream();\r\n" +
        "        System.setOut(new PrintStream(output));\r\n" +
        "        %s\r\n" +
        "        return output.toString();\r\n" +
        "    }\r\n" +
        "}";

private final String className;
private final String javaSource;

public DcpExecutor(String className, String javaSource) {
    this.className = className;
    this.javaSource = String.format(sourceSkeleton, className, javaSource);
}

public byte[] compile() {
    JavaMemoryCompiler compiler = new JavaMemoryCompiler();
    JavaFileObject javaObject = new JavaMemoryObject(className, javaSource);
    List<Class<? extends Object>> classPath = new ArrayList<Class<? extends Object>>();
    classPath.add(Executable.class);

    return compiler.compile(javaObject, classPath);
}

JavaMemoryObject

This class serves two purposes:
  • it contains the source Java code that will be compiled. I pass this object to the Compiler object to read the source from member javaSource;
  • it contains the compiled Java byte code (in ByteArrayOutputStream). The Compiler calls the openOutputStream method and writes the byte code in it;
public class JavaMemoryObject extends SimpleJavaFileObject {

    private final String javaSource;
    private final ByteArrayOutputStream bos = new ByteArrayOutputStream();

    public JavaMemoryObject(String className, String javaSource) {
        super(getUri(className, Kind.SOURCE), Kind.SOURCE);
        this.javaSource = javaSource;
    }

    public JavaMemoryObject(String className, Kind kind) {
        super(getUri(className, kind), kind);
        this.javaSource = null;
    }

    public byte[] getBytes() {
        return bos.toByteArray();
    }

    @Override
    public CharSequence getCharContent(boolean ignoreEncodingErrors) throws IOException {
        return javaSource;
    }

    @Override
    public OutputStream openOutputStream() throws IOException {
        return bos;
    }

    private static URI getUri(String className, Kind kind) {
        return URI.create("string:///" + className.replace('.', '/') + kind.extension);
    }
}

JavaMemoryCompiler

This class performs the actual compilation. It uses the standard Java Compiler API from ToolProvider.

It uses a special JavaMemoryManager (detailed below) to trigger the writing of the resulting byte-code into a JavaMemoryObject.

Note also the conversion of class path elements from Iterable<Class> to a String containing a semicolon (;) delimited sequence of .class or .jar files, which contain the actual resource named in the Iterable.

public byte[] compile(JavaFileObject javaObject, Iterable<Class<?>> classPaths) {
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    JavaMemoryManager manager = new JavaMemoryManager(compiler.getStandardFileManager(null, null, null));
    Iterable<JavaFileObject> javaObjects = Arrays.asList(javaObject);

    List<String> options = new ArrayList<String>();
    if (classPaths != null) {
        options.add("-cp");
        String classPath = buildClassPath(classPaths);
        options.add(classPath);
    }

    CompilationTask task = compiler.getTask(null, manager, null, options, null, javaObjects);
    if (!task.call()) { // compile error
        throw new RuntimeException("Compilation error");
    }

    return manager.getBytes();
}

private String buildClassPath(Iterable<Class<?>> classPaths) {
    Set<String> pathSet = new HashSet<String>();
    for (Class<?> clazz : classPaths) {
        pathSet.add(getCompilationPath(clazz));
    }
    StringBuilder classPathBuilder = new StringBuilder();
    for (String path : pathSet) {
        classPathBuilder.append(path).append(";");
    }

    return classPathBuilder.toString();
}

private String getCompilationPath(Class<?> clazz) {
    String className = clazz.getName();
    className = className.replace('.', '/') + ".class";
    URL classUrl = getClass().getClassLoader().getResource(className);
    String filePath = classUrl.getPath();

    try {
        int exclamationIndex = filePath.indexOf("!");
        if (exclamationIndex >= 0) { // is jar
            filePath = filePath.substring(0, exclamationIndex);
            classUrl = new URL(filePath);
        } else { // is class
            int extensionIndex = filePath.lastIndexOf(className);
            filePath = filePath.substring(0, extensionIndex);
            classUrl = new URL(classUrl.getProtocol(), classUrl.getHost(), filePath);
        }

        File classFile = new File(classUrl.toURI());
        String path = classFile.getPath();

        return path;
    } catch (Exception e) {
        log.error("Exception occurred", e);
    }

    return null;
}

JavaMemoryManager

This class is a wrapper around the JavaMemoryObject. The compiler calls this class' getJavaFileForOutput method when it's about to write the generated byte code into the .class file. Instead of a real file, this implementation uses a JavaMemoryObject that contains a byte array stream. Therefore, the entire compiled byte array is written into this stream, which can be accessed later.

Hence, all the compilation happens in-memory -- without any files being written to the file system.

public class JavaMemoryManager extends ForwardingJavaFileManager<StandardJavaFileManager> {

    private JavaMemoryObject javaMemoryObject;

    protected JavaMemoryManager(StandardJavaFileManager fileManager) {
        super(fileManager);
    }

    @Override
    public JavaFileObject getJavaFileForOutput(Location location, String className, Kind kind, FileObject sibling)
            throws IOException {
        javaMemoryObject = new JavaMemoryObject(className, kind);

        return javaMemoryObject;
    }

    public byte[] getBytes() {
        return javaMemoryObject.getBytes();
    }
}


Comments

Popular posts from this blog

Running sp_updatestats on AWS RDS database

Part of the maintenance tasks that I perform on a MSSQL Content Manager database is to run stored procedure sp_updatestats . exec sp_updatestats However, that is not supported on an AWS RDS instance. The error message below indicates that only the sa  account can perform this: Msg 15247 , Level 16 , State 1 , Procedure sp_updatestats, Line 15 [Batch Start Line 0 ] User does not have permission to perform this action. Instead there are several posts that suggest using UPDATE STATISTICS instead: https://dba.stackexchange.com/questions/145982/sp-updatestats-vs-update-statistics I stumbled upon the following post from 2008 (!!!), https://social.msdn.microsoft.com/Forums/sqlserver/en-US/186e3db0-fe37-4c31-b017-8e7c24d19697/spupdatestats-fails-to-run-with-permission-error-under-dbopriveleged-user , which describes a way to wrap the call to sp_updatestats and execute it under a different user: create procedure dbo.sp_updstats with execute as 'dbo' as

Content Delivery Monitoring in AWS with CloudWatch

This post describes a way of monitoring a Tridion 9 combined Deployer by sending the health checks into a custom metric in CloudWatch in AWS. The same approach can also be used for other Content Delivery services. Once the metric is available in CloudWatch, we can create alarms in case the service errors out or becomes unresponsive. The overall architecture is as follows: Content Delivery service sends heartbeat (or exposes HTTP endpoint) for monitoring Monitoring Agent checks heartbeat (or HTTP health check) regularly and stores health state AWS lambda function: runs regularly reads the health state from Monitoring Agent pushes custom metrics into CloudWatch I am running the Deployer ( installation docs ) and Monitoring Agent ( installation docs ) on a t2.medium EC2 instance running CentOS on which I also installed the Systems Manager Agent (SSM Agent) ( installation docs ). In my case I have a combined Deployer that I want to monitor. This consists of an Endpoint and a

Debugging a Tridion 2011 Event System

OK, so you wrote your Tridion Event System. Now it's time to debug it. I know this is a hypothetical situtation -- your code never needs any kind of debugging ;) but indulge me... Recently, Alvin Reyes ( @nivlong ) blogged about being difficult to know how exactly to debug a Tridion Event System. More exactly, the question was " What process do I attach to for debugging even system code? ". Unfortunately, there is no simple or generic answer for it. Different events are fired by different Tridion CM modules. These modules run as different programs (or services) or run inside other programs (e.g. IIS). This means that you will need to monitor (or debug) different processes, based on which events your code handles. So the usual suspects are: dllhost.exe (or dllhost3g.exe ) - running as the MTSUser is the SDL Tridion Content Manager COM+ application and it fires events on generic TOM objects (e.g. events based on Tridion.ContentManager.Extensibility.Events.CrudEven