Scale or die

To organize our internal IT information we have been using twiki. It is a very flexible tool by virtue of being a wiki and has two critical features out of the box that other wikis seem to lack:

1. Forms
2. and a fairly interesting search directive (%SEARCH%)

If you are not using these with twiki you are missing out; the analogy is using Word without styles. You can do without but life is so much easier with them.

Forms brings structure to wiki pages and allow to treat wiki pages as a structured record (the form) with a big, free-form description field (the page). For instance our twiki implementation records hosts, hardware items, services, change requests, incident tickets all with the use of custom forms, so as to produce a pseudo-relational database on which we build reports.

Examples of reports:

1. list of all change requests awaiting peer review before approval
2. list of all hosts assigned to a given project
3. list of all hosts running on a given piece of hardware

The list goes on. Then we start having questions such as “which are the hardware pieces whose leases end in the next 3 months?” or “how many hosts run RedHat 4.5?”. And that is when twiki breaks… Its reliance on a file-based scheme (and rcs) to maintain relationship imposes some unwelcome limitations, not to mention a level of performance that is difficult to accept on a daily basis (I know that caching is in the works but it is just not built to scale).

Case in point: we define hosts (think linux hosts) as compute resources that execute on some physical substrate (think IBM x3550) so it is only natural that the host form has a mention for the hardware item it executes on. In other words there is a one-to-n relationship between hardware item and host. On the hardware item form we do not feature the list of hosts that live on that hardware item because chances of dangling pointers are too great. We used to have it and quickly we ended up with hosts that point to a piece of hardware, which itself does not point back to these hosts.

In other words we have had to limit the type of reports we can run because the underlying data implementation of twiki is lacking. Questions such as “Which hardware items are home to more than 3 hosts?” become unnecessarily complicated, whereas with the proper framework it becomes as simple as:

select hi.name, count(*)
from hardware_item hi
join harware_host ho on (hi.sid = ho.hardware_sid)
group by hi.name
having count(*) >= 3


How about the list of potential single points of failure for a given service:
select max(h.name) as hostname, hc.name as host_class
from service s
join service_host so on (s.sid = so.service_sid)
join host h on (so.host_sid = h.sid)
join host_class hc on (h.class_sid = hc.sid)
where s.name = “My critical service”
group by hc.name
having count(*) < 2;


Now, assuming I have such a relational database that twiki can query via a sql module, how different is it from the database that my monitoring package is based upon? In the ideal world my data model presents something that:

• monitoring can use (service dependencies, host maps, etc.)
• configuration management can use (change requests bound to given hosts, software items, etc.)
• asset management can use
• finance can use

The key properties that I would want such a system to keep are the ease of use with which it be manipulated (nothing more cumbersome that twiki) and its accuracy (no duplicate data). At the same time I have not found any product out there fits the bill (a monitoring package that has a solid data model that be extended for other uses). So I might just bite the bullet and build a prototypical ERP for IT.

Stay tuned.