Skip to main content

Executing in Memory a JSP DCP from Database

By Unknown
This is follow-up of my previous post Executing a JSP DCP Stored in the Database, where I was presenting a proof-of-concept on executing a string representing a JSP DCP stored in the Content Delivery Database.

I wasn't too happy with the previous solution, because it was writing the compiled .class to the file system and there was no caching at all (the .class would be recompiled with every request). I can do better than that! :-)

So the new version is compiling everything into a memory byte array, then executes the in-memory class. The compiled class is placed into an LRU cache, in order to optimize performance.

The entire example is available on Google code project; below are just the highlights:

Custom JSP Tag

This is the calling code that initiates the execution of the DCP.

public void doTag() throws JspException {
    try {
        DcpClassLoader loader = new DcpClassLoader(componentUri, componentTemplateUri);
        Executable dcpExecutable = loader.getExecutable();
        String result = dcpExecutable.execute();

        JspContext context = getJspContext();
        JspWriter out = context.getOut();
        out.write(result);
    } catch (Exception e) {
        throw new JspException(e);
    }
}

DcpClassLoader

The class extends ClassLoader and is resposible for finding dynamic in-memory class for each DCP. It uses an LRU cache to store the recently created dynamic DCP classes. It also takes into account the lastPublicationDate of the DCP, which is used to invalidate the cache.

This class also instantiates the retrieved class (an implementation of the Executable interface).

If the DCP class is not found in the cache, this is the entry point into the in-memory compiler (method createExecutorClass).

public DcpClassLoader(String componentUri, String componentTemplateUri) throws ParseException {
    super(DcpClassLoader.class.getClassLoader());

    cp = getComponentPresentation(componentUri, componentTemplateUri);
    lastModified = getLastPublicationDate(cp);
    className = String.format("Dcp_%d_%d_%d", cp.getPublicationId(), cp.getComponentId(),
            cp.getComponentTemplateId());
}

@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
    ClassLoaderCache cache = ClassLoaderCache.getInstance();
    CacheEntry cacheEntry = cache.get(name);
    if (cacheEntry != null && cacheEntry.getLastModified() > lastModified) {
        return cacheEntry.getClazz();
    }

    Class<?> clazz = createExecutorClass(name);
    cache.put(name, clazz);

    return clazz;
}

public Executable getExecutable() throws ClassNotFoundException {
    try {
        Class<?> clazz = loadClass(className);
        return (Executable) clazz.newInstance();
    } catch (Exception e) {
        log.error("Exception occurred", e);
    }

    return null;
}

private Class<?> createExecutorClass(String name) throws ClassFormatError {
    DcpExecutor executor = new DcpExecutor(name, cp.getContent());
    byte[] classBytes = executor.compile();

    return defineClass(name, classBytes, 0, classBytes.length);
}

DcpExecutor

This class puts the DCP content into a Java class context (the sourceSkeleton), compiles it in memory, and returns the byte code of the compiled class.

The sourceSkeleton is in fact implementing the Executable interface (which only defines a public String execute() method). In order to compile the dynamic Java source, the resource mitza.dynamic.compile.Executable needs to be made available to the compiler (in the classpath). Therefore, the compiler accepts an Iterable<Class> containing the classPath elements.

private static final String sourceSkeleton =
        "import java.io.ByteArrayOutputStream;\r\n" +
        "import java.io.PrintStream;\r\n" +
        "import mitza.dynamic.compile.Executable;\r\n" +

        "public class %s implements Executable {\r\n" +

        "    public String execute() {\r\n" +
        "        ByteArrayOutputStream output = new ByteArrayOutputStream();\r\n" +
        "        System.setOut(new PrintStream(output));\r\n" +
        "        %s\r\n" +
        "        return output.toString();\r\n" +
        "    }\r\n" +
        "}";

private final String className;
private final String javaSource;

public DcpExecutor(String className, String javaSource) {
    this.className = className;
    this.javaSource = String.format(sourceSkeleton, className, javaSource);
}

public byte[] compile() {
    JavaMemoryCompiler compiler = new JavaMemoryCompiler();
    JavaFileObject javaObject = new JavaMemoryObject(className, javaSource);
    List<Class<? extends Object>> classPath = new ArrayList<Class<? extends Object>>();
    classPath.add(Executable.class);

    return compiler.compile(javaObject, classPath);
}

JavaMemoryObject

This class serves two purposes:
  • it contains the source Java code that will be compiled. I pass this object to the Compiler object to read the source from member javaSource;
  • it contains the compiled Java byte code (in ByteArrayOutputStream). The Compiler calls the openOutputStream method and writes the byte code in it;
public class JavaMemoryObject extends SimpleJavaFileObject {

    private final String javaSource;
    private final ByteArrayOutputStream bos = new ByteArrayOutputStream();

    public JavaMemoryObject(String className, String javaSource) {
        super(getUri(className, Kind.SOURCE), Kind.SOURCE);
        this.javaSource = javaSource;
    }

    public JavaMemoryObject(String className, Kind kind) {
        super(getUri(className, kind), kind);
        this.javaSource = null;
    }

    public byte[] getBytes() {
        return bos.toByteArray();
    }

    @Override
    public CharSequence getCharContent(boolean ignoreEncodingErrors) throws IOException {
        return javaSource;
    }

    @Override
    public OutputStream openOutputStream() throws IOException {
        return bos;
    }

    private static URI getUri(String className, Kind kind) {
        return URI.create("string:///" + className.replace('.', '/') + kind.extension);
    }
}

JavaMemoryCompiler

This class performs the actual compilation. It uses the standard Java Compiler API from ToolProvider.

It uses a special JavaMemoryManager (detailed below) to trigger the writing of the resulting byte-code into a JavaMemoryObject.

Note also the conversion of class path elements from Iterable<Class> to a String containing a semicolon (;) delimited sequence of .class or .jar files, which contain the actual resource named in the Iterable.

public byte[] compile(JavaFileObject javaObject, Iterable<Class<?>> classPaths) {
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    JavaMemoryManager manager = new JavaMemoryManager(compiler.getStandardFileManager(null, null, null));
    Iterable<JavaFileObject> javaObjects = Arrays.asList(javaObject);

     List<String> options = new ArrayList<String>();
    if (classPaths != null) {
        options.add("-cp");
        String classPath = buildClassPath(classPaths);
        options.add(classPath);
    }

    CompilationTask task = compiler.getTask(null, manager, null, options, null, javaObjects);
    if (!task.call()) { // compile error
        throw new RuntimeException("Compilation error");
    }

    return manager.getBytes();
}

private String buildClassPath(Iterable<Class<?>> classPaths) {
    Set<String> pathSet = new HashSet<String>();
    for (Class<?> clazz : classPaths) {
        pathSet.add(getCompilationPath(clazz));
    }
    StringBuilder classPathBuilder = new StringBuilder();
    for (String path : pathSet) {
        classPathBuilder.append(path).append(";");
    }

    return classPathBuilder.toString();
}

private String getCompilationPath(Class<?> clazz) {
    String className = clazz.getName();
    className = className.replace('.', '/') + ".class";
    URL classUrl = getClass().getClassLoader().getResource(className);
    String filePath = classUrl.getPath();

    try {
        int exclamationIndex = filePath.indexOf("!");
        if (exclamationIndex >= 0) { // is jar
            filePath = filePath.substring(0, exclamationIndex);
            classUrl = new URL(filePath);
        } else { // is class
            int extensionIndex = filePath.lastIndexOf(className);
            filePath = filePath.substring(0, extensionIndex);
            classUrl = new URL(classUrl.getProtocol(), classUrl.getHost(), filePath);
        }

        File classFile = new File(classUrl.toURI());
        String path = classFile.getPath();

        return path;
    } catch (Exception e) {
        log.error("Exception occurred", e);
    }

    return null;
}

JavaMemoryManager

This class is a wrapper around the JavaMemoryObject. The compiler calls this class' getJavaFileForOutput method when it's about to write the generated byte code into the .class file. Instead of a real file, this implementation uses a JavaMemoryObject that contains a byte array stream. Therefore, the entire compiled byte array is written into this stream, which can be accessed later.

Hence, all the compilation happens in-memory -- without any files being written to the file system.

public class JavaMemoryManager extends ForwardingJavaFileManager<StandardJavaFileManager> {

    private JavaMemoryObject javaMemoryObject;

    protected JavaMemoryManager(StandardJavaFileManager fileManager) {
        super(fileManager);
    }

    @Override
    public JavaFileObject getJavaFileForOutput(Location location, String className, Kind kind, FileObject sibling)
            throws IOException {
        javaMemoryObject = new JavaMemoryObject(className, kind);

        return javaMemoryObject;
    }

    public byte[] getBytes() {
        return javaMemoryObject.getBytes();
    }
}


Labels:

Comments

Post a Comment

Popular posts from this blog

Running sp_updatestats on AWS RDS database

By
Part of the maintenance tasks that I perform on a MSSQL Content Manager database is to run stored procedure sp_updatestats . exec sp_updatestats However, that is not supported on an AWS RDS instance. The error message below indicates that only the sa  account can perform this: Msg 15247 , Level 16 , State 1 , Procedure sp_updatestats, Line 15 [Batch Start Line 0 ] User does not have permission to perform this action. Instead there are several posts that suggest using UPDATE STATISTICS instead: https://dba.stackexchange.com/questions/145982/sp-updatestats-vs-update-statistics I stumbled upon the following post from 2008 (!!!), https://social.msdn.microsoft.com/Forums/sqlserver/en-US/186e3db0-fe37-4c31-b017-8e7c24d19697/spupdatestats-fails-to-run-with-permission-error-under-dbopriveleged-user , which describes a way to wrap the call to sp_updatestats and execute it under a different user: create procedure dbo.sp_updstats with execute as 'dbo' as...
2 comments
Read more

REL Standard Tag Library

By
The RSTL is a library of REL tags providing standard functionality such as iterating collections, conditionals, imports, assignments, XML XSLT transformations, formatting dates, etc. RSTL distributable is available on my Google Code page under  REL Standard Tag Library . Always use the latest JAR . This post describes each RSTL tag in the library explaining its functionality, attributes and providing examples. For understanding the way expressions are evaluated, please read my post about the  Expression Language used by REL Standard Tag Library . <c:choose> / <c:when> / <c:otherwise> Syntax:     <c:choose>         <c:when test="expr1">             Do something         </c:when>         <c:when test="expr2">             Do something else         </c:when...
2 comments
Read more

Publish Binaries to Mapped Structure Groups

By Unknown
Today's TBB of the Week comes from the high demand in the field to publish binary assets to different mapped Structure Groups. By default SDL Tridion offers two ways of publishing binaries: All binaries publish to a folder defined in your Publication properties; All binaries rendered by a given template publish to a folder corresponding to a given Structure Group; In my view, both cases are terrible, over-simplified and not representing a real use-case. Nobody in the field wants all binaries in one folder and nobody separates binary locations by template. Instead, everybody wants a mapping mechanism that takes a binary and publishes it to a given folder, defined by a Structure Group, and this mapping is done using some kind of metadata. More often than not, the metadata is the TCM Folder location of the Multimedia Component. I have seen this implemented numerous times. So the solution to publish binaries to a given location implies finding a mapping from a TCM Folder to a...
10 comments
Read more