Yet Another Tridion Blog

This post describes how to convert the 'sdl-healthcheck' application to a Windows Service. SDL-healthcheck is a Java application that creates a webservice around the SDL Web microservices in order to provide monitoring (heart beat) checks on them. Mark van der Wal wrote the application and provided startup and stop Shell scripts. My requirement was to have the same application installable and executable as Windows Service. Therefore, I had to wrap the Java application inside a service container and provide start, stop hooks for the service. I know that SDL Web Content Delivery microservices are also Java applications using Spring Boot, so my approach was to use the same mechanism to make the Java application run as Windows Service. The service container used is 'procrun.exe' daemon from Apache Commons . Depending on the arguments passed to it, it can perform different Windows Service functionality such as install, remove service, start/stop service. I ended ...

In order to squeeze the most performance out of the publishing in a Tridion, SDL Web system, there is a high amount of parameters and configurations that must all be optimized. This post presents a few of these parameters, especially in the context of AWS infrastructure. CME The Tridion GUI, the Content Manager Explorer server is a normal Tridion installation, but with Publisher and Transport services disabled. This server is only used for running the website -- the CME. Publishers Publisher servers are Tridion installations without the CME. Namely these servers have the Publisher and Transport services enabled and any other Tridion services, if installed, are disabled. The purpose of a Publisher is simply to lookup publishable items in the Publishing Queue and proceed to render them, hand them over to the Transport service and have the transport package pushed to the Content Delivery. Publishers should use approximately twice the number of CPU cores as number of renderi...

When a Deployer Receiver receives a transport package, it notifies the Deployer Workers there is 'work' for them to do. This notification can take the form of JMS messages sent using some kinds of queuing mechanism. This post describes the Amazon Simple Queuing Service (SQS) to send these notifications. The Deployer workers receive these messages from SQS and they start deploying/undeploying the package. In order to setup this notification system, we must first create the SQS queues and configure them across the Deployer Receiver and all Deployer Workers. Start by creating Standard Queues using all default properties. We need 3 queues (commit, content, and prepare): Once the SQS queues are in place, we can configure the Deployers to use them in the file  deployer-config.xml . Amazon gives us the queues URLs. We need to specify the URL base separately, and then simply name the queues individually, as per below: <Queues> <Queue Default= "true...

When a Deployer Receiver receives a transport package, it notifies the Deployer Workers there is 'work' for them to do. This notification can take the form of files on a shared File System. The Deployer workers monitor the file system and upon noticing a change, they start deploying/undeploying the package. In order to setup this File System notification system, we must first create a shared File System (shared across the Deployer Receiver and all Deployer Workers). In order to do that, have a look at an earlier post Using Elastic File System for Out-Scaled Deployers . Once the EFS is in place, for example under folder /efs01 , we can configure the Deployers to use this share file system in the file deployer-config.xml , as per below: <Queues> <Queue Default= "true" Verbs= "Content" Adapter= "FileSystem" Id= "ContentQueue" > <Property Name= "Destination" Value= "/efs01/deployer-queu...

In a scaled-out scenario for Content Delivery Deployers, it is possible to setup an AWS ElastiCache Redis DB as the Binary Storage medium for incoming transport packages. The Deployer Receiver writes these transport package zip files into the Binary Storage Redis instance. Then it is up to the Deployer Workers to read these zip files as they deployer/underploy the content. Below, we present the configurations for an AWS ElastiCache Redis acting as storage medium for transport packages. Start by simply creating an ElastiCache Redis instance in AWS console. This whole step might take you 5 minutes :) AWS will give you the hostname where the Redis DB is available and the port, usually 6379. In your deployer-conf.xml , setup this Redis instance using the BinaryStorage node format as below:     <BinaryStorage Id="RedisStorage" Adapter="RedisBlobStorage">         <Property Name="Host" Value="10.10.2.232"/>       ...

In a scaled-out scenario for Content Delivery Deployers, it is possible to setup a shared file system as the Binary Storage medium for incoming transport packages. The Deployer Receiver writes these transport package zip files into the Binary Storage folder. Then it is up to the Deployer Workers to read these zip files as they deployer/underploy the content. Below, we present the configurations for an AWS Elastic File System (EFS) acting as storage medium for transport packages. Start by simply creating an EFS in AWS console. This whole step might take you 5 minutes :) AWS is going to generate a hostname where this file system is available and it will give instructions on how to mount it in your server. For example, in Linux CentOS, one can mount an FS using the mount command. The following command will mount the EFS drive under folder /efs01 on the current server: sudo mount -v -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-...

This post is part of a bigger topic Autoscaling Publishers in AWS . In a previous post, I mentioned the LifeCycle Termination Hooks for our Auto Scaling Policy . In this post, we see more details about this hook and how it is actually used to gracefully shutdown service on the instance that is about to be terminated. As per earlier post, we defined one termination hook in the Auto Scaling Policy, named 'sdl_terminate_publisher': Next, we use this a CloudWatch event to execute a Lambda Function that performs the graceful shutdown of the Publisher service on the instance, and then releases the termination hook, so the instance can be terminated properly. In CloudWatch, create a new Rule as per below: Event Source: Event Pattern Based on service: Auto Scaling Event Type: Instance Launch and Terminate Specific event: EC2 Instance-terminate Lifecycle Action Specific group: sdl_publisher-asg Target a Lambda function to be executed when this event triggers: SDL...

This post is part of a bigger topic Autoscaling Publishers in AWS . In a previous post we talked about the Auto Scaling Groups , but we didn't go into details on the Scaling Policies. This is the purpose of this blog post. As defined earlier, the Scaling Policies define the rules according to which the group size is increased or decreased. These rules are based on instance metrics (e.g. CPU), CloudWatch custom metrics, or even CloudWatch alarms and their states and values. We defined a Scaling Policy with Steps, called 'increase_group_size', which is triggered first by the CloudWatch Alarm 'Publish_Alarm' defined earlier. Also depending on the size of the monitored CloudWatch custom metric 'Waiting for Publish', the Scaling Policy with Steps can add a difference number of instances to the group. The scaling policy sets the number of instances in group to 1 if there are between 1000 and 2000 items Waiting for Publish in the queue. It also sets the ...

This post is part of a bigger topic Autoscaling Publishers in AWS . Now that we have a Launch Configuration , based on it, we can create an Auto Scaling Group. This will be in charge of creating/terminating publisher instances based on some rules (Scaling Policies). We base our Auto Scaling Group on the Launch Configuration created earlier (sdl_publisher_lc). The very important properties are the Desired, Min and Max number of instances in the group. The Desired is set usually by the Scaling Policies, but more about those later. The Min and Max are the limits of this group. In my case I use Min=1 and Max=3, meaning that I want to have 1 publisher running at all times and when needed, based on load, 2 additional publishers can be added to the group by an 'Increasing size' Scaling Policy. Once load passes, a 'Decreasing size' Scaling Policy reduces the number of instances in the group. Scaling Policies These policies represent the rules for adding/removing in...

This post is part of a bigger topic Autoscaling Publishers in AWS . Tridion Publishers need a license that matches the name of the host machine they run on. Each Publisher must have a different name. This means that when using autoscaling, we must know in advance which license to copy into the instance (at first start up time) and also we must rename that machine's hostname to match the license. This is needed because otherwise the instance machine name is always the one set in the AMI. The script below executed by the Amazon Launch Configuration service when the publisher instance starts. It is backed by a database that contains the available licenses, each with a host name. The script takes the first license available and then changes the host name to match the name specified in the license. Additionally, the script then downloads the correct license files from their s3 location and copies them in the right folder under the Tridion installation. Finally the script instruc...

This post is part of a bigger topic Autoscaling Publishers in AWS . Each Auto Scaling Group in AWS is based on a Launch Configuration. The Launch Configuration cannot be modified once it's created, but it can however be copied and modified. Once Auto Scaling is in place, the instances it creates will all be based on the specified Launch Configuration. Creating a Launch Configuration implies specifying the AMI, the Instance Type, Security IAM role, EBS drive, security group and open incoming ports. All these are AWS settings and they don't make the object of this post. What is particularly interesting is the section Configure details , subsection User data . This is where one can specify either a script of a file containing a script that will be executed inside the instance once it is created and it starts. Using this mechanism we created PowerShell scripts to configure each Publisher instance in part. <powershell> aws s3 cp s3://mybucket/scripts/publisher_user...

This post is part of a bigger topic Autoscaling Publishers in AWS . Autoscaling works based on some metrics. For Tridion Publishers we are going to define a Publish_Alarm  in CloudWatch that uses the custom metric Waiting for Publish  in order to trigger the alarm or not. The alarm defines a threshold that, if passed, sets its state to ALARM. Otherwise, if the metric is below, it sets the state to OK. Based on these states, one can define the Scaling Policies of an Auto Scale Group (more about this in a later post). We are going to create an alarm that monitors the custom metric Waiting for Publish . In the first screen of creating the alarm, we select this metric. On the second screen, we specify the threshold and the duration it takes to consider this alarm triggered. For example, I used a threshold of 1000 items that has to be surpassed for 3 consecutive readings (each reading happening at 1 minute interval). We leave the Actions section empty for now. We...

This post is part of a bigger topic  Autoscaling Publishers in AWS . In order to define autoscaling of some servers, we need some metrics that we can use to create the autoscaling logic, i.e. when to spin up new instances and when to terminate them. A good measure for this, in Tridion terms, is the size of the Publishing Queue. Namely for Publishers autoscaling, it's useful to look at the number of items in the Publishing Queue that are in the state "Waiting for Publish". The approach is to read this metric somehow from the Tridion Content Manager database and make it available in AWS, so that we can use it later. AWS CloudWatch provides a way to define and/or intercept events that can trigger some code execution. The code executed is supposed to read the Publishing Queue and push the count of items into CloudWatch as a custom metric. 1. Define Lambda Function This function represents the code that is executed by the CloudWatch rule. The function reads the size of...

This series of blog posts presents in detail the setup of Tridion Publishers autoscaling as implemented for Amazon Web Services (AWS) infrastructure. Originally, I was inspired by Julian's post about  Using AWS to Scale SDL Web Publishers . I used this post to have an initial setup and then I completed it with more features like license management and more monitoring. These are the steps that I took: Publishing Queue metrics in CloudWatch CloudWatch alarms Launch Configuration License management Auto Scaling Group Scaling Policies Terminate Lifecycle Hook

Part of the maintenance tasks that I perform on a MSSQL Content Manager database is to run stored procedure sp_updatestats . exec sp_updatestats However, that is not supported on an AWS RDS instance. The error message below indicates that only the sa  account can perform this: Msg 15247 , Level 16 , State 1 , Procedure sp_updatestats, Line 15 [Batch Start Line 0 ] User does not have permission to perform this action. Instead there are several posts that suggest using UPDATE STATISTICS instead: https://dba.stackexchange.com/questions/145982/sp-updatestats-vs-update-statistics I stumbled upon the following post from 2008 (!!!), https://social.msdn.microsoft.com/Forums/sqlserver/en-US/186e3db0-fe37-4c31-b017-8e7c24d19697/spupdatestats-fails-to-run-with-permission-error-under-dbopriveleged-user , which describes a way to wrap the call to sp_updatestats and execute it under a different user: create procedure dbo.sp_updstats with execute as 'dbo' as...

This post describes a backup/restore procedure for the Content Manager database in an Amazon Web Services (AWS) environment). The database is a MSSQL instance running on as Amazon Relational Database Service (RDS). The existing database is from an SDL Tridion 2013 environment, and as such, after the restore in the new SDL Web 8.5 environment, it will have to be upgraded to SDL Web 8.5. Performing the backup implies taking the database offline and executing the stored procedure: exec msdb.dbo.rds_backup_database 'Tridion_cm_2013' , 'arn:aws:s3:::mybackup/Tridion_cm_2013.bak' The procedure is described in http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html Performing the restore implies executing the following stored procedure in the new RDS database: exec msdb.dbo.rds_restore_database 'Tridion_cm_2013' , 'arn:aws:s3:::mybackup/Tridion_cm_2013.bak' , '' While the restore is running, you ...

In a recent project, we had the requirement to install the SDL Web 8.5 Content Manager and the Publisher using script only (aka an unattended installation). I knew about the existence of such an unattended installation, but I for one, have never attempted it. The feature is clearly documented in http://docs.sdl.com/LiveContent/content/en-US/SDL%20Web-v5/GUID-CE873235-5FE0-489D-A63C-B979919D8F9E . All the prerequisites must still be fulfilled before the actual unattended installation can take place: In Server Manager: Server Roles: Web Server (IIS) Features: .NET Frameowrk 4.6 ASP.NET 4.6 WCF Services (all of them, including other features needed as dependencies) Web Server Role (IIS) Role Services Common HTTP Features (all of them) Health & Diagnostics (all) Performance (all) Security (all) Install Java Runtime Environment Content Manager DB must be created in advance. MTS user must be created in advance. Once all prerequisites are met, I was...