Showing posts with label TADDM. Show all posts
Showing posts with label TADDM. Show all posts

Monday, February 14, 2011

A new fix pack for TADDM - 7.2.0.0-TIV-ITADDM-FP0004

A new fix pack is out for TADDM. This one doesn’t have anything really major, just lots of bugfixes.

http://www-01.ibm.com/support/docview.wss?uid=swg24028518

A lot of them seem to center around Out Of Memory and Storage Errors during discovery or just having the discovery engine or console hanging.

Unless you’ve got specific problems, there’s no reason to get excited over this one.

On the other hand, TADDM 7.2.1, which will lots and lots of NEW shiny features, should be coming out quite soon.

I wonder if anyone is (a) reading this and (b) as excited as I am about it :)

-- Robert

Tuesday, July 13, 2010

Extended Attributes Sensor

When using TADDM, you will probably come across a case where you want to extract information which TADDM does not do out of the box.

For example, you want to record the date on which the antivirus was last updated on each server.

The extended attributes feature enables you to add a field to those TADDM recognizes. The Custom Server option enables you to run OS commands and scripts and populate them.

http://www.ibm.com/developerworks/wikis/display/tivoliaddm/Sensor+Examples

However, writing a script that extracts the data can be time consuming – this isn’t what we want to be doing. Populating the extended attributes is also a bit more complex than populating regular ones.

Fortunately, there are a number of shortcuts. The DeveloperWorks TADDM wiki has a sensor which will read a config file, create the extended attributes and populate them when TADDM discovers the relevant computer. All you need to do is enter the attribute name and the registry key (or WMI object, etc…) in the config file.

Extended Attributes Sensor and Tools

I used it today and when it runs, it’s a big timesaver.

However, I did have to tweak a few things for it to run on TADDM 7.2FP1 on Windows. In case this helps you, here is what I did:

  • in the file ea_cli.bat, I changed the line
    @../support/bin/jython_coll.bat ea_cli.jy %*
    to
    @../support/bin/jython_coll.bat ea_cli.jy %*
  • in the file sensorhelper.py, I changed the line
    from com.ibm.cdb.platform.ip import IpUtils
    to
    from com.ibm.cdb.platform.ip import IPv4Utils
  • in the file ext_attr_db.py, I changed the line
    from com.ibm.cdb.topomgr.util import JdbcConnectionFactory
    to
    from com.collation.topomgr.util import JdbcConnectionFactory

Without the first change, nothing works. The second and third changes are because I got the error “ImportError: no module named ” when I ran the script.

Another useful sensor is this generic sensor.

-- Robert

Wednesday, March 24, 2010

TADDM 7.2 Fix Pack 1 – current list of APARs

It’s not due till the middle of April, but details have just been posted about the APARs fixed in TADDM 7.2FP1.

This fix pack is scheduled to release April 12, 2010. This is a target date and does not represent a formal commitment by IBM. This date is subject to change without notice.

I’ve listed a few of the APARs that seem interesting (to me) or that will solve my problems. The full list is in the link.

APAR IZ62734
When the TaddmTool has a newer version of TaddmWmi.dll an internal error occurred while copying the file from the gateway to the target. This was happening because the old TaddmWmi.dll is still loaded.

APAR IZ63361
When discovering a lot of targets, the building topology phase can hang or run extremely slow. The TopologyBuilder.log shows that ComputerSystemConsolidation agent is running for very long time. Topology garbage collector log shows that topology JVM memory is used up and garbage collector cannot release anything.

APAR IZ63682
When the Vmware sensor discovers Computer System with a different primary IpInterface, it does not merge with an already existing ComputerSystem in the database causing duplicates to occur. This is happening because the TopologyBuilder Agent uses only existing Computer systems without virtual flag set to true.

APAR IZ64198
In change history the date/time were displayed as UTC milliseconds from the epoch. The Date/time were fixed to be displayed in a more human readable way.

APAR IZ64526
Modules from IIS Web Server discovery were not included in business application (using both app descriptor and UI). IIS was not following a standard like other web servers IPlanet or Apache.

APAR IZ64682
Out of memory errors can occur on Windows platforms where an MS Exchange application is installed and contains many Public Folders. This was due to the large amount of data returned from the system.

APAR IZ68913
The component that sends change events to Omnibus or TEC -- changeevents.sh or changeevents.bat -- looks for the last change event indicator file in a different directory from where it was written after last discovery. Because it does not find the file, the change events program sends ALL the change event records, even those sent previously. This symptom negatively affects performance. The following entry appears in ChangeEvent.log: [main] INFO changeevent.OmpEventModule - Could not locate change events run time indicator at C:\ibm\cmdb\dist\var/services\change-events-runtime.log (This is ok in TADDM versions prior to 7.2).

APAR IZ69885
Populating the extended attribute for template extension using a jython script does not appear in UI. Additionally in the error.log the following message appears:
TopologyBuilder [TopologyBuilderEngineThread] ERROR agents.ExtendedAttributesAgent - [ExtendedAttributesAgent.E.2] An error occurred when setting extended attributes for ComputerSystem XXXXXXXXXXXXXXX

APAR IZ70822
Due to changes for Launch in Context following query http://<taddm_server_IP>:9430/cdm/queryHomePage.do?launchsource=itm&searchtext=<compsysIP> stopped working because it was filtered out by security mechanism. This behaviour will be enabled to handle correctly queries from other products.
APAR IZ71582
The following Launch in Context entries stopped working because it was filtered out by security mechanism. This behavior will be enabled to handle correctly queries from other products.
"http://$taddm_server$:$taddm_port$/cdm/changehistory.do?objectID=guidChanges$taddm_guid$&amp;domainIP=localhost&amp;pageID=domain&amp;hoursback=240"
"http://$taddm_server$:$taddm_port$/cdm/changehistory.do?objectID=guidChanges$taddm_guid$&amp;domainIP=localhost&amp;pageID=domain&amp;hoursback=240"

 

Robert

Wednesday, February 17, 2010

TADDM 7.2 Interim Fix 1

Get the first fix of the year for TADDM here.

There are only two bugs/solutions in this interim fix, both concerning upgrades from a previous version of TADDM.

From the readme:

IZ67686 TADDM upgrade on a Windows platform failed during the database migration.
The error occurs when TADDM is installed in a different location than default C:\IBM\CMDB
The Customer may find this message in the taddm_7.2.0_install_msg.log:
INFO : com.ibm.cdb.install.ia.utils.Utils (from runCommand(content, envs, filetype)) - Stdout:
"Run command E:\ibm\cmdb\dist\bin\migration.bat -s -bv 7.1.2"
INFO : com.ibm.cdb.install.ia.utils.Utils (from runCommand(content, envs, filetype)) -
Stderr: 'migration.bat' is not recognized as an internal or external command, operable
program or batch file.

IZ68633 If the following files are missing: collation/linux.zip, collation/solaris.zip, collation/aix.zip,
collation/windows.zip, collation/linuxS390.zip the install/migration reports successful, but TADDM will
not start because the jdks needed to run TADDM and move the internal users from the 7.1.2 encryption
to 7.2 encryption are not available. The TADDM installation will be missing the
file: dist/etc/ TADDMSec.properties.


All I can say is that these are boring bugs! ;)



Robert

Saturday, February 6, 2010

Pinging the Unpingable

Despite it’s name, the TADDM PingSensor does not run the same ICMP ping that the command line utility ping does. What the PingSensor does is open a TCP socket on specific ports.
The list of ports used is in the Installation Guide:Planning Worksheet:

Port Target
CiscoWorks 1741
DNS 53
LDAP 389
SSH 22
WBEM 5988
WMI 135

Other ports used are 23 and 161.

This means that if a firewall is blocking these ports, then TADDM will not see any device there, even if another port, such as 80, is open. In that case, you’ll see an error message in the PingSensor log:

2010-02-06 14:13:11,875 DiscoverManager [DiscoverWorker-14] PingSensor-64.12.100.5 INFO session.Ping - Ping failed for IP address 64.12.100.5 on all ports [22, 23, 135, 161]

In this case, you’ll either need to <gasp> open the firewall between TADDM and that server or add another port to the list TADDM scans by adding a property to collation.properties.

From the sensor settings section of the Administration Guide: com.collation.pingagent.ports=xx,yy, ...

By default, this property is used by the PingSensor sensor. It is not defined in the collation.properties file and has to be manually defined if needed.
Valid values are non-negative, numerical.
To override the default set of ports that the Ping sensor attempts to use, add this property to the collation.properties file and specify the port numbers as a comma-separated list. The default set of ports the Ping sensor attempts to use are port 22 and then port 135, if it cannot first make a connection to port 22.
For example, to add the SNMP port 161 to the existing ports that the Ping sensor attempts to connect to, you would add 161 to the end of the list of default ports:
com.collation.pingagent.ports=22,135,161.
If you only wanted the Ping sensor to use port 161, set it to the following: com.collation.pingagent.ports=161.


Sunday, January 31, 2010

Bleeding edge isn’t what is should be

Looks like there’s a problem with TADDM using the new 1.7.1 version of Cygwin as it’s SSH server.

http://www-01.ibm.com/support/docview.wss?uid=swg21419038&myns=swgtiv&mynp=OCSSPLFC&mync=R

So if you’re downloading Cygwin for a TADDM installation, use the link to the older, 1.5 version of Cygwin - http://cygwin.com/setup-legacy.exe 

You could actually just rename the regular setup.exe to setup-legacy.exe and it will download the latest patch level of the older version of Cygwin.

Monday, April 13, 2009

Reusing computer names - TADDM discovery

When a server is virtualized, i.e. converted from a physical server into a virtual server, then TADDM had a problem.
Because these are different computers, that merely share the same name (fully qualified domain name - FQDN), they show up twice in the inventory list.

Worse, the old server exists in all the business applications, but the new one doesn't. I'd have to manually update all the applications with the new servers.

Fortunately, TADDM 7.1.2 Interim Fix 1 has just been released. Among it's improvements there is this:

IZ44531 Reusing fully qualified domain names causes operating systems to be shared among old and
new computer systems. In 712, one can now prevent OS FQDN from being set by using the following collation.properties setting:
com.collation.discover.agent.ComputerSystemAgent.ignoreFQDN
If property does not exist in collation.properties, the default value is taken.
Default value is false. If the customer set it to true, FQDN will not be set for the OS.


Property can be configurated through the OS or IP:
com.collation.discover.agent.ComputerSystemAgent.ignoreFQDN.Linux.1.2.3.4
or
com.collation.discover.agent.ComputerSystemAgent.ignoreFQDN.Linux
or
com.collation.discover.agent.ComputerSystemAgent.ignoreFQDN.1.2.3.4
Where Linux is the OS name and 1.2.3.4 is the IP.


Example for ignoring FQDN on VMWare operating systems:
com.collation.discover.agent.ComputerSystemAgent.ignoreFQDN.Vmnix=true

Note that the os.fqdn is not unset if it is already there. This change only applies to NEW operating systems.
To find operating systems that share computer systems, use this query;
./dbquery.sh "select GUID_X, PK__OSRUNNING_X, LASTMODIFIEDTIME_X,
CONTEXTIP_X, SIGNATURE_X from COMPSYS where PK__OSRUNNING_X IN (SELECT
comp.PK__OSRUNNING_X from COMPSYS comp, OPSYS op where
comp.PK__OSRUNNING_X = op.GUID_X || '-0' and
op.PK__PARENTOPERATINGSYSTEM_X != comp.GUID_X || '-0') order by
PK__OSRUNNING_X, LASTMODIFIEDTIME_X"


The api delete command can be used to remove the computer system which is no longer valid.

Some other interesting tidbits in the new fix:

  • REST Api
    New API to allow TADDM data and functionality to be used over
    HTTP with JSON and XML. See "Developing applications using the REST API" in
    the SDK Developer's Guide for more information. Very interesting
  • Add copy capability to details panel in the TADDM GUI console. Needed this from day 1!
  • IZ39662 The installer should fail and cause a FATAL error if the files required for
    installation are not there. This was my bug!
  • IZ41245 The TADDM installer should remove obsolete files that can cause errors in the logs. This is something all applications should always do - how many times have we seen old and unneeded files, configuration items and miscellaneous objects in older software?

Good luck with the new version :)

-- Robert

Wednesday, March 25, 2009

Troubleshooting guides for TBSM, ITM and TADDM

A few links that are useful, collected together:

ITM:

ITM 6.2.1

ITM 6.2 Fixpack 1

ITM 6.2

ITM 6.1 (Revised)

TADDM:

TADDM 7.1.2

TADDM 7.1

TBSM:

TBSM 4.2

TBSM 4.1.1

TBSM 4.1

-- Robert

Tuesday, March 3, 2009

Resetting wasadmin password

I recently inherited a TADDM environment where I needed to work with WebSphere in order to integrate TADDM with Active Directory.

Unfortunately, no one knew the password for wasadmin, so I couldn't access the Websphere Admin's console. Time was pressing, the other guy was not available and my knowledge of WebSphere administration is limited to what's written in the TADDM and TBSM documentation (hint: very little!)

I followed the following routine. It worked for me.

  1. I needed to configure WebSphere not to use security:
    1. I found the file security.xml under /cell directory.
    2. I edited it, so that under the security:Security tag I changed enabled="true" to enabled="false"
  2. Now, I needed to restart the WebSphere so it would reset it's security.
  3. The following is the "proper" routine to stop WebSphere:
    1. <WAS_HOME>/profiles/ctgAppSrv01/bin/stopServer.sh MXServer -username wasadmin -password wasadmin
    2. <WAS_HOME>/profiles/ctgAppSrv01/bin/stopNode.sh -username wasadmin -password wasadmin
    3. <WAS_HOME>/profiles/ctgDmgr01/bin/stopManager.sh -username wasadmin -password wasadmin
  4. Of course, not having the password meant I couldn't stop the processes. Well, that's what kill -9 is for...
  5. Starting WebSphere is easier:
    1. <WAS_HOME>/profiles/ctgDmgr01/bin/startManager.sh
    2. <WAS_HOME>/profiles/ctgAppSrv01/bin/startNode.sh
    3. <WAS_HOME>/profiles/ctgAppSrv01/bin/startServer.sh MXServer
  6. Now, when I access the Integrated Solutions Console (http://server:9060/ibm/console/login.do) then I'm no longer asked for a password
  7. I login as wasadmin and recreate all the security I need...

I think (hope) this is more-or-less what a WebSphere expert would do too!

-- Robert

Monday, January 19, 2009

What is TADDM stuck doing?

Yesterday I ran a discovery in an environment and after the sensors finished scanning the servers, the topology builder started it's work.

2%..4%..6%..9%..11%...75%.

It breezed through the first percentiles quickly and smoothly, but stuck on 75% for ages. Eventually we left and came back the next day.
The good news is that it completed successfully.

I don't like not knowing what and why computers are doing something. Why did it get stuck on 75% but nowhere else?
The percentiles that the topology builder show us are now percentiles such as displayed when copying a file. I.e. 9% doesn't mean that 9% of the work has been done, it means that 4 "steps" have been completed. So TADDM was stuck on the step represented by 75%.

per this Note, the various steps are represented in the file $COLLATION_HOME/dist/etc/TopologyBuilderConfigurationDefault.xml and there are nearly 40 steps.

Each step represents the topology builder generating the dependencies and links for a sensor. Part of creating a brand-new sensor is updating this file, so that after the discovery is done the topology builder will fold the items found by the new sensor into the list of links it holds.

Here are the sensors and the percentile they are represented by:

2 DatabaseServerCleanupAgent
4 ComputerSystemConsolidationAgent
6 ComputerSystemTypeAgent
9 RuntimeGcAgent
11 DeletedObjectGcAgent
13 SystempConsolidationAgent
16 VmwareVirtualCSConsolidationAgent
18 J2EEServerDeploymentAgent
20 AppServerClusterAgent
23 JDBCDependencyAgent
25 JBossClusterAgent
27 WeblogicClusterAgent
30 OracleAppClusterAgent
32 OracleDependencyAgent
34 WebConnectionDependencyAgent
37 WebSphereConnectionDependencyAgent
39 SAPDependencyAgent
41 SoftwareHostReferenceAgent
44 DNSServiceAgent
46 LDAPServiceAgent
48 DNSDependencyAgent
51 NFSDependencyAgent
53 HostDependencyAgent
55 ConnectionDependencyAgent
58 GenericAppAgent
60 AppDefinitionAgent
62 AppDescriptorAgent
65 ObjectDisplayNameAgent
67 DiscoveryLogCleanupAgent
69 DominoConnectionAgent
72 DominoClusterDomainAgent
74 PortableAgentWrapper
76 l2.L2Agent
79 l2.CDPAgent
81 CompositeCreationAgent
83 MQServerAgent
86 CitrixAgent
88 ExchangeDependencyAgent
90 VCSDependencyAgent

Seeing as I was stuck on 75% I see that the L2 (communications layer 2) sensor was very busy. In retrospect this makes prefect sense, since this discovery was the first time I found a new type of switch in the environment. It makes sense that the builder would take much longer for this step than when I'm only discovering servers.

What happens after step 90%?

I don't know. I'll try to find out and let you know!
If anyone does know, please drop a line

-- 2011 Update

In TADDM 7.2.x, the log file dist/log/TADDM.log details each step, so if your discovery is stuck on step x, just check the last line of the log and you'll see what class the Topology Builder is working on.

-- Robert

Wednesday, January 14, 2009

TADDM OMP Change Event Module on TADDM 7.1.2

The TADDM OMP Change Event Module (http://www-01.ibm.com/software/brandcatalog/portal/opal/details?catalog.label=1TW10CC1Q) is a module you install on TADDM servers so that you will get alerts on various monitoring systems when a change (any change or specific changes) are detected.

This module works very well on TADDM 7.1, but it fails on TADDM 7.1.2.


The error message is:
com.collation.proxy.api.client.ApiException: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.UnmarshalException: invalid method hash
at com.collation.proxy.api.client.DataApiImpl.findModel(DataApiImpl.java:954)

Apparently there were changes in the Java files between 7. 1 and 7.1.2.

The solution is to change following line in /bin/changeevents.sh:

omp_classpath=.:"$TADDM_PATH"/lib/platform-model.jar:"$TADDM_PATH"/lib/api-client.jar:"$TADDM_PATH"/lib/api-dep.jar:"$TADDM_PATH"/lib/edm-server.jar:"$TADDM_PATH"/lib/oal-api.jar:"$TADDM_PATH"/lib/oal-common.jar:"$TADDM_PATH"/lib/oal-topomgr.jar:"$JAR_DIR"/evd.jar:"$JAR_DIR"/log.jar:"$JAR_DIR"/taddmomp.jar

to

omp_classpath=.:$TADDM_PATH/sdk/clientlib/taddm-api-client.jar:"$JAR_DIR"/evd.jar:"$JAR_DIR"/log.jar:"$JAR_DIR"/taddmomp.jar

Hope this helps whoever encounters this problem.

 

-- Robert

Sunday, November 9, 2008

New versions...

I guess it's just that time of the year again!

TBSM 4.2 came out recently, as did Impact 5.1.

Yesterday I saw both TADDM 7.1.2 and ITM 6.2.1 in passport advantage.

Here's a brief list of TADDM changes from the release notes:

TADDM 7.1.2 gives you the rich details of configuration items with automated, agentless discovery of the assets and their application dependencies, as well as a Discovery Library technology to help leverage data from other sources.

TADDM is a configuration management tool that helps IT operations personnel ensure and improve application availability in application environments. The operational staff gets a top-down view of applications so the staff can quickly understand the structure, status, configuration, and change history of their business-critical applications. This view immediately isolates issues in times of performance or availability problems and enables more effective planning for application change without disruption. An agent-free creation and maintenance of a Configuration Management Database (TADDM database) is delivered without requiring custom infrastructure modeling. TADDM also provides complete cross-tier dependency maps, topological views, change tracking, event propagation, and detailed reports and analytics.

The following list includes the functionality that was added for the TADDM 7.1.2:

  • BIRT Report Infrastructure
  • Limited IPV6 Capability
  • Console Installation Capability
  • Improved View Performance
  • Improved Details Performance
  • Improved ECMDB synchronization times
  • Improved API query performance
  • Improved post-discovery processing performance
  • Improved TBSM Integration
  • Drill Down Capability for Business Applications
  • Comparison Report From Domain Manager
  • Cross-Domain Comparison reports from ECMDB
  • Additional MQ Cluster Information for zOS
  • Reduced WAN traffic during anchor usage
  • Upgraded first failure data capture tools
  • Weblogic 9.x and 10.x sensor support
  • Simplified migration from previous releases
  • Windows 2008 Support
  • AIX 6.1 Support
  • Bug Fixes

Updated documentation is here.

ITM 6.2.1 doesn't seem to have updated the documentation yet, but here's a list I have of it's changes:

    • Adaptive Monitoring
    • TEPS Changes
      • Event Slot Customization - replaces my older post!
      • Managed node search bar
      • Zoom in charts
      • SSO with Java Webstart
    • SPB bundles for TCM/TPM
    • More CLI abilities
      • Replace wadminep function (getfile, put file, list dir, execfile)
      • CLI for historical data configuration collection
      • Remotely invoke pdcollect tool
      • Expand tacmd createsit (display item, consecutive samples, state)
    • Agent Builder
      • Support 100+ connections for remote monitoring
      • Browser for logfile/script monitors
      • Add CIM provider
    • Out of the box Agentless OS monitoring packages
    • Infrastructure improvements
      • 64-bit zLinux and AIX support
      • Support TDW on zOS
      • Support TEC events from z Hub
      • Manage agent fail-over
      • Asynchronous Deployment
      • Support 64 bit counters
      • Agent Manager Services - an extra agentlet who manages the regular agents
    • Improved TADDM integration

Of those, the Adaptive Monitoring and Event Slots seem extremely interesting and I really want to try them out!

-- Robert

Edit: forgot a few things in ITM...

Wednesday, October 15, 2008

How shall I TADDM the network, if I don't know what it's like?

TADDM, which I've written before, is IBM's detective. It sniffs out and find everything that's connected to the network, interrogates them to find out what's inside and maps out the relationships (who's talking to whom and and who's ignoring whom).

Now, the first time I have to remove the dewy-eyed look from users is when I inform them that, yes, TADDM can discover their unique 3rd-party/in-house applications, BUT... we have to tell TADDM what these products look like so that it can "tag" them correctly. They want TADDM to be able to "guess" or even divine which of the thousands of processes and files scattered around the network are related to which business service they run. TADDM can do that, but it need at least a little help!

The other thing they don't like is when I have to feed TADDM the TCP/IP address scopes and user/passwords of their servers. They say "Why do I need to enter all these things if TADDM is supposed to discover everything by itself?". I must then reassure them that they do not want TADDM to be able to hack into their secure systems without the right credentials!
OK, so TADDM requires the users/passwords for the servers (it can also get the passwords from the organizations LDAP repository) but why does it need me to enter the IP addresses?

For two good reasons: (1) So that  you can give each IP list (scope in TADDM jargon) it's own set of credentials, i.e. IP 200.100.10.1 through 200.100.10.255 uses a certain user/password and a another scope uses a different one. (2) So that you can schedule discoveries by scope, and that way you're not doing the whole net each time.

But... what happens when an organization doesn't know how to split up it's IP range into separate environments? Say there are 200 servers, split into development, test, production and DRP environments all overlapping in the same IP scope. If I tried running discovery, I'd get tons of errors for using wrong user/passwords and I'd get a lot of unneeded devices when all I want is the production environment, unless I matched up IP to environment first.

Of course, the whole point of TADDM is that you don't necessarily know your network ahead of time. So what do you do?

The solution is a bit iterative, but you're not running into errors on the way and you don't need to feed TADDM any knowledge about the way the network is mapped ahead of time. What happens is that TADDM creates something called Logical Connections between devices when it detects that one uses the other. Logical Connections can be between application servers and databases, web servers and storage area network devices, switches and ... you get the idea). Now, once TADDM has completed a discovery, it maps out the Logical Connections between items which lie within it's scope. LC which go beyond the scope are listed, but not displayed on the map.

To leverage this, what we do is get our nose under the tent - we discover a few servers we know are in the right environment, find out which servers they are connected to, add them to the scope and continue till we have discovered everything in the environment we are interested in.

There are two ways of listing Logical Connections outside of the TADDM GUI:

  1. ./api.sh -u username-p password find LogicalConnection
    api.bat for Windows users.
  2. Connect to the database and run select distinct fromip_x, toip_x from logcconn

The first command will create an XML file with all the data, the second regular SQL. I used the second command (slightly modified to ignore 127.0.0.1 and output the data nicely) to create a script file which runs loadscope after the select and then I get an ever expanding scope which contains items which are talking to each other.

Don't forget that loadscope looks for a file formatted like this:

IP_Address/range/subnet, Exceptions, Description 
IP_Address/range/subnet, Exceptions, Description 
IP_Address/range/subnet, Exceptions, Description

Thanks to Byron for the help :)

Hmph.... This post seems a little too long for what I wanted to say... I hope I was clear in the end.

-- Robert

Saturday, September 20, 2008

Reports

It's all very well to have a display of what is going on in your system at the moment - but what's been happening during the course of last month? Can you compare last year to today? How can I prove that whatever-change-was-just-made has (or has not) made a difference?


Sure, you can hack together an SQL routine or Perl script to get the raw data out of the monitoring system, but then what about showing your conclusions to someone who doesn't speak your type of jargon? You need something which creates reports which have been made for human eyes - not man/machine hybrids.

In other words, you need reports so you can translate your technical knowledge into business knowledge and in that way share your IT information with the decision making sections of your company/organization.

 

image 

TCR1

image

One of the nicer ideas in Tivoli at the moment is the gradual merging of all the various reporting routines in the myriad Tivoli products.

IBM has taken the standard BIRT reporting system and wrapped it up as Tivoli Common Reporting (TCR) - all the cooler newer versions of the Tivoli family have their reports in this new standard. The site I linked to has a list of all the TCR offerings. More and more of them are published on OPAL all the time. ITM6.2 reports have just had an update, for example.

Using BIRT means that the reporting engine is (a) free and (b) easily customizable - for those who know what they're doing with it. Alas, I'm not yet quite good enough with BIRT to create my own extra-special reports.

The next version of TADDM will use these reports and I'm curious as how to go about creating a "mashup" of a report which merges CMDB data with monitoring events - for example, how about a report which shows Number of Failures as a function of Number of Configuration Changes across the organization?

-- Robert

Thursday, July 31, 2008

TADDM logs and logging tips

A good TADDM technote has been published. It lists the various logs and their meanings.
The logs are located in $COLLATION_HOME\log. I've bolded the more useful ones.

control.log - contains messages from the start script

cdm.log - web portal logs are located here

discover.log - contains messages from the Discover jini service

discover-admin.log - contains messages from the DiscoverAdmin jini service

error.log - contains serious errors for any of the TADDM/CDT services

events-core.log - contains messages from the events core jini service

l2.log - contains messages for the layer2

local-anchor.<hostname>.log - logs messages from J2EE sensors

login.log - user login audit trail

proxy.log - contains messages from the Proxy jini service

tomcat.log - contains messages from the startup sequence

topology.log - contains messages from the topology jini service

services/ApiServer.log - XML, Java and EJB interfaces to CMDB are processed here

services/ChangeManager.log - ChangeManager works with StateManager to process change events after discovery completes

services/ClientProxy.log - start here for GUI issues. The GUI talks to client proxy exclusively

services/DiscoverManager.log - contains message for Sensor and Template Matcher messages (see comment further on)

services/DiscoverObserver.log - moves completed work items from DiscoverManager to TopologyManager

services/MonitorStateManager.log - processes discovery and change events

services/ProcessFlowManager.log - event processing engine for Discovery

services/ReportsServer.log - Handles some reports tasks

services/TopologyManager.log - interface between all other components and the datastore.

services/ViewManager.log - ViewManager builds the CI navigation trees

If the setting com.collation.discover.engine.SplitSensorLog is set to TRUE in the /etc/collation.properties then each sensor will have it's own log in the directory log/sensors/<runid>/sensorName-IP.log. this is VERY useful in debugging discovery errors. If the value is set to FALSE then the DiscoveryManager will contain all the messages running together. I always make sure the SplitSensor is set to TRUE.

Here are some more log file settings which are defined in the /etc/collation.properties file

  • com.collation.log.level - Logging level. Default is INFO, I set it to DEBUG whenever I need to see why a discovery failed. At DEBUG level you'll see the precise taddmtool command which was run and the server's response.
  • com.collation.deploy.dynamic.logging.enabled - If true, you don't have to restart TADDM when you change logging settings. I always make sure this is true, the (slight) performance drop you (might) get due to the constant rechecking is worth it when you want to debug a discovery.
  • com.collation.log.filesize - Controls the maximum size of each log file. Default is 20 Megabytes. When is limit is reached, the file is renamed and a new file is started.
  • com.collation.log.filecount - Maximum number of log files created (older ones are deleted). Default is 3.

-- Robert

Wednesday, June 25, 2008

Using Extended Attributes in TADDM with the TADDM API

As I've mentioned before, TADDM is the heart of IBM's CMDB solution.

Now, each Configuration Item (CI) in the CMDB has a very long list of attributes; For a computer it's the computer's name, IP address, amount of memory and so on. For a database it's the list of schemas, physical files and so on.

Now, TADDM is smart and can extract just about everything possible about a system. However, there's no way in the world it can know about outside information - such as the name of the person in charge of the specific server. That's why there are a number of attributes which are not discovered automatically, but are entered manually. These are called administration attributes are mainly names, phone-numbers and so on.

Now, you may have a use for information of this sort that does not appear as one of the administration attributes. The classic example is the physical location of the server. To solve this problem, TADDM has Extended Attributes which you can create and add to any type of CI. You could add a PhysicalLocation field to every ComputerSystem and a NeedsToBeBackedUp field to every database.

The only problem is that the default reports will not show you your Extended Attributes.

There are two solutions at the moment (I understand that the default reporting options should improve by the end of this year).

The smart solution is to download the TADDM connector for TDI which will enable you to manipulate your TADDM database easily. (After you've gone through the learning curve, of course)

The simple(r) solution is to write a short Java program which will extract the information for you.

I recommend you do both - and see what works best for you. You'll certainly learn more about TADDM this way.

Here is a short Java program which will demonstrate how to use the Java API to get data out of TADDM


----- Begin ShowAttributes.java

// I don't think you need ALL these includes for this specific program to run, but they never hurt...
import com.collation.platform.model.Guid;
import com.collation.platform.model.AttributeNotSetException;
import com.collation.platform.model.ModelObject;
import com.collation.platform.model.topology.extattrib.UserData;
import com.collation.platform.model.topology.meta.UserDataAttributeMeta;
import com.collation.platform.model.topology.meta.UserDataMeta;
import com.collation.platform.model.topology.sys.ComputerSystem;
import com.collation.platform.model.topology.sys.OperatingSystem;
import com.collation.platform.model.topology.sys.windows.WindowsOperatingSystem;
import com.collation.platform.util.ModelFactory;
import com.collation.proxy.api.client.ApiConnection;
import com.collation.proxy.api.client.ApiException;
import com.collation.proxy.api.client.ApiSession;
import com.collation.proxy.api.client.CMDBApi;
import com.collation.proxy.api.common.AttrNameValue;
import com.ibm.cdb.api.ApiFactory;
import java.io.*;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Vector;

public class ShowAttributes{

        String usage = "Usage: ShowAttributes <hostname> <port> <username> <password> <condition>";
        if (args.length < 6) {
            System.out.println(usage);
            System.exit(1);
        }
        String hostname = args[0];
        int port = Integer.parseInt(args[1]);
        String username = args[2];
        String password = args[3];
        String condition = args[4];
        String extendedAttribute = args[5];

        //We've got our initial arguments. Lets access the TADDM database!
        CMDBApi Api= null;
        ApiSession Session = null;
        ApiConnection Conn = ApiFactory.getInstance().getApiConnection(hostname, port, null, false);
        Session = ApiFactory.getInstance().getSession(Conn, username, password, ApiSession.DEFAULT_VERSION);
        Api = Session.createCMDBApi();

        //We'll extract a list of ComputerSystems
        Query = "SELECT * FROM ComputerSystem where fqdn contains '" + condition + "'";            
        ModelObject mo[] = Api.find(Query, -1, null, null);
        System.out.println("Found " + String.valueOf(mo.length) + " items."); 
       
        //Loop over the list of computers
        for (int j=0; j<mo.length; j++) {   

              //Print out the computer's name
              ComputerSystem computer = (ComputerSystem) mo[j];
              System.out.println(computer.getDisplayName()); 
                                          
              //Extract the list of Extended Attributes for our specific computer
              Guid guid = mo[j].getGuid();
              AttrNameValue[] attrNameValGet = api.getExtendedAttributes(guid);
              //Loop over the list of Extended Attributes  
              for (int jj=0; jj<attrNameValGet.length; jj++) { 
                    
                          System.out.println(">" + attrNameValGet[jj].name + "," + attrNameValGet[jj].value + ","); 

               }

         } 
}

---- End ShowAttributes.java

(This has been cut & pasted without re-double-triple-checking, so please tell me if there are problems)

Now, the problems don't end just with writing the program. How do you compile and run it?
Java being Java, you need a ton of parameters so that the right jar files will be found.
Here's a batch file which compiles our program and runs it.

---- Begin ShowAttributes.bat

@echo off
rem You must set these two to suit your environment.
set COLLATION_HOME=D:\IBM\cmdb\dist
set JAVA_HOME=%COLLATION_HOME%\external\jdk-1.5.0-Windows-i386

rem We will now set TADDM's environment so we can compile java
set path=%path%;%JAVA_HOME%\bin
set TADDM_SDK_HOME=%COLLATION_HOME%\sdk
call %TADDM_SDK_HOME%\bin\setTaddmEnv.bat

echo Compiling ShowAttributes.java
javac -Xlint:deprecation -cp %TADDM_CLASSPATH% -d . ShowAttributes.java
if %errorlevel% neq 0 exit /b
Rem Continue if there were no compiler errors

java -Dcom.collation.home=%COLLATION_HOME% -cp %COLLATION_HOME%\lib\api-client.jar;%COLLATION_HOME%\lib\api-dep.jar;%COLLATION_HOME%\lib\platform-model.jar;. ShowAttributes %1 %2 %3 %4 %5 %6 %7 %8

pause

---- End ShowAttributes.bat

Read more about the TADDM API and other TADDM tricks and tips with the new Redbooks which have just been published.

IBM Tivoli Application Dependency Discovery Manager Capabilities and Best Practices

Deployment Guide Series: IBM Tivoli Application Dependency Discovery Manager V7.1

Have fun!