Tutorial CDE

Inteligencia de Negocio y Pentaho: Cómo hacer cuadros de mando: V: Un tutorial recomendado para manejar las excelentes herramientas que Pedro Alves y Webdetails han desarrollado, en especial CDE.

Your faithful employee

Hello,

In the area I work for, we have one duty among several others more: Maximize the availability of the network management systems. The NMSs must run almost every time because the NOCs (network operation centers) monitor them 7x24.

But thats not an easy task, sometimes the planning area delivers the NMS implementation with many flaws, other times the machine you get is not what you expected or the application servers freezes continously and research the root cause can take weeks.

As we're not big fan of attending service disruption calls at 3am, we deployed a nice and useful service in all our Linux machines: Monit. This nice program monitor the service existence,availability and performance and take automated actions when the rules/thresholds are exceeded.

For example, we had a Tomcat container that was getting frozen several times at the week...the rule for Monit was something like this:

check process tomcat5 with pidfile /var/run/tomcat5.pid
group tomcat5
start program = "/etc/init.d/tomcat5 start" with timeout 120 seconds
stop  program = "/etc/init.d/tomcat5 stop" with timeout 120 seconds
if failed host 127.0.0.1 port 8080 
protocol HTTP request /archivos/gestion.jpg 
TIMEOUT 3 SECONDS then restart
if cpu usage > 95% for 10 cycles then restart
if 5 restarts within 5 cycles then timeout

So, this loyalty automated employee check the port 8080, check the HTTP request of gestion.jpg to be less than 3 seconds and check the cpu usage of the process to be under 95%. If Monit sees any of these rules broken then it begins to restart the service. As the good employee he is, Monit sends email notification of every step he takes.

Hope this application be useful for you,


Quick update: Even thought Monit is doing a great job is important to find the root cause. Regarding the tomcat issue i found this useful site to tune the JVM parameters: http://wiki.alfresco.com/wiki/JVM_Tuning

pentaho CDE CDF CDA CD* tutorials

Quick post to help people trying to make stuff on those great tools developed by webdetails.pt . They are great but have a lack of docs, so this two links would help:

http://www.vinzi.nl/media/CDE-introduction_V0_8.pdf
http://www.tikalk.com/incubator/blog/creating-bugzilla-dashboard-%E2%80%93-hands-cde-tutorial-%E2%80%93-fuse-day-3-session-summary

I got those links in IRC ##pentaho channel on freenode btw

PD: If you want to install CDE in a context path different to /pentaho/, keep in mind that uri is hardcoded in all js and templates files of the pentaho-cde-dd directory

Messing with Roo

In the last Google I/O was anounced that GWT 2 will have support in Spring Roo. I never heard of Roo before and I wanted to take look (http://www.springsource.org/roo).

It seems that Roo improves the development by maintaining all the interfaces, stubs and glue code that is boring when you use GWT or Entities.

I just ran two of the demos...a classic Spring Controller Roo Project with handling the CRUD of a Concat entity (http://s3.springsource.com/MRKT/roo/2010-01-Five_Minutes_Roo.mov):

And a GWT integration handling the CRUD of an equipment (http://www.thescreencast.com/2010/05/how-to-gwt-roo.html)

So far  it was pretty impressive to make an entity based application in 5 minutes without all the glue code, but as this is a new concept I wonder if anyone has done any real large sized project supported in Roo....

Render a Prpt report correctly in a Pentaho's BI Platform xaction

Hi, a quick hint about writing a Xaction who must render a Prpr Report. If you try to use the xaction writing plugin for Eclipse, the xaction probably wont work. Here is the correct xaction, and you should overwrite the xaction with this in Eclipse (changing the prpt name of course):

<?xml version="1.0" encoding="UTF-8"?>
<action-sequence>
  <title>Test report in PDF format</title>
  <version>1</version>
  <logging-level>TRACE</logging-level>
  <documentation>
    <author>Emmanuel Guiton</author> 
    <description>Generates a PDF test report.</description> 
    <help/> 
    <result-type/> 
    <icon/>
  </documentation>
  <inputs/>
  <outputs>
    <test_report type="content">
      <destinations>
        <response>content</response>
      </destinations>
    </test_report>
  </outputs>
  <resources>
    <report-definition>
      <solution-file>
        <location>test-report.prpt</location> 
        <mime-type>application/zip</mime-type>
      </solution-file>
    </report-definition>
  </resources>
 
  <actions>
    <action-definition>
      <component-name>SimpleReportingComponent</component-name>
      <action-type>Test report</action-type>
      <action-resources>
        <report-definition type="resource"/>
      </action-resources>
      <action-outputs>
        <outputstream type="content" mapping="test_report"/>
      </action-outputs>
      <component-definition>
        <outputType><![CDATA[application/pdf]]></outputType> 
        <useContentRepository><![CDATA[true]]></useContentRepository> 
        <paginate><![CDATA[false]]></paginate>
      </component-definition>
    </action-definition>

  </actions>
</action-sequence>

Hope this help.

Latest CDF - script.js minified

Hi,
Just a quick post about the latest CDE-CDF etc that the guys of webdetails have been developing. As In the previous version, distributed in Pentaho platform, I had to tweak a little the javascript of the Map component, I wanted to do the same in this version.

You'll notice a new js file called script.js. That's a minified js generated when the server is started. If you need to twek the js and test you'll have to add this to the dashboard's url: &debug=true&bypassCache=true

Hope this helps anyone else

Aberrant Behavior Detection

As I wrote before, most of the time of this year I worked in Network Management Systems. Managing a huge, constantly growing, network is a heavy task. One part of this task is measurement collection, and for this task we use Cacti, which a powerful visualization and rrd graphics template tool.

Cacti as others open source measurement collections systems rely the data storing and charting in the RRDTool software.

In our Cacti instances we are collecting measurements from 15000 different router/cmts interfaces, which includes common measures like bandwidth, cpu, memory and others not so common like SNR, Cable Modems in the upstream, modulation profile.

As the NOC is unable to watch and control this amount of charts, there is a nice functionality, not widely used, in the rrds: Aberrant Behavior Detection (ABD). ABD adds a set of measurements and algorithms that adapt to the seasonal trend of the chart and detects when a current value is too deviated from the expected value. You can read about the mathematical details here and here.

You can see some examples here:

The classic bandwidth measurement

Cable Modems in the upstream channel

You can write a short Perl script to check if there is an aberrant behavior in the rrd and do something about about it (for example, send a trap or a syslog with the alert). This way you add a great feature to the network management tasks.