Monitoring Cluster-Enabled Quartz Instances

From AgileApps Support Wiki

This process sets up an automatic check that sends an email alert when Quartz is down.

  1. Verify that these lines are in the AALSchedulers.xml file. (They are present in new installations, but are not automatically added during upgrades.)
    <attribute name="org.quartz.scheduler.instanceId" = value="AUTO"/>         # Automatically assigns instance IDs to prevent conflicts
    ...
    <attribute name="org.quartz.jobStore.misfireThreshold" = value="60000"/>   # Prevent infinite loop if a job can't start, but keeps retrying
    ...
    <attribute name="org.quartz.jobStore.isClustered" = value="true"/>         # Turn on clustering
    <attribute name="org.quartz.jobStore.clusterCheckinInterval" = value="20000"/>
    
  2. Download the monitor_quartz.pl (Perl) script from the platform downloads area.
  3. Set these variable values for your installation:
    • $mysql_username - Database user name.
    • $mysql_password - Database password.
    • $mysql_host - Name of server the database is running on.
    • $config_file - Absolute path to tomcat/webapps/networking/WEB-INF/web.xml
    • $recipients - Comma-separated list of email addresses who will receive email status messages
  4. Run the script in the background.
    You'll get an email alert whenever the script discovers that Quartz isn't running.
How it Works
The lines in the properties file cause quartz to put an entry into the relationals.QRTZ_SCHEDULER_STATE table in the database. The script checks the timestamp to verify that the last stamp was within 5 minutes. (Quartz should be running once a minute. If it hasn't run in five minutes, it is considered to be down.)