The RTMC Project.
(Real Time Modular Collection)
General Questions:
RTMC Questions:


General Questions:

Q:
Can RTMC do anything else besides collect metrics?
A: Yes. The function of RTMC is really defined by the plugins it runs. Meaning the engine that is the RTMC executable does not place restrictions on what plugins can and can not do. Currently the plugins being developed for RTMC collect system metric data, but nothing is preventing another developer from taking RTMC and turning it into a mail server or a cryptography key breaker. In fact it can do both at the same time.

One of the hands-off design aspects many developers have is multi-threading or running parallel tasks. It is a taboo some developers have; akin to the goto statement in C. Just mentioning adding multi-threading to an application and you can see the weight being added to developers shoulders. So what I did was create an application that was dynamically multi-threaded (not a set of static known threads) and allow drop-in plugin support. That way once I achieved this goal I decided to use it for something specific; in this case metric gathering.

Q:
Why are you writing a system metric tool for Linux when there are so many alternatives already out there?
A: There are two answers to this question. The first is as a professional I work at the enterprise level of development and most widely use metrics gathering tools are at the local OS level or have to be queried at the device driver level. So gathering system metrics data from the enterprise point of view is painfully difficult or expensive; e.g., as in money. The task of bridging tools that were not developed with enterprise collection in mind is a paintful task of ad-hoc middleware for each tools out there too. This leads to skewed metrics and a mess of long term maintenance issues from maintaing code to third party licenses. So it's a good problem to tackle.

The second, I wanted something to show off some of my talents as an engineer. When you work under NDA contracts and sometimes layoffs are inevitable cycle of business decisions that have no bearing on ones performance, recruiters and human resource manangers have a hard time finding real talent based on a few keywords on a piece of paper. RTMC is something real to show interested parties besides diplomas and years of service.

Q:
What's the difference between RTMC and {fill in the blank} metric tool?
A: Most system metric tools are very static in how you the developer or system administrator can get to the data; i.e., mostly via the console. There are some tools that provide the metric data as a fixed stream or from an API if you are willing to pay for commerical products. Either way integrating the tools to fit your solution is usually difficult. RTMC abondons the traditional software design model system metric tools follow, and creates dynamic way for metric data gathering.

RTMC is 100% multi-threaded. That means processor, network, diskio, memory, etc. etc. data collection is performed in parallel. To gain a real snapshot of what is happening on a system you have to gather this data close to the same time. RTMC was built from the ground up with plugin support in mind. This means RTMC is not limited to collecting only a specific set of metrics, it can collect any metric you want without needing to write a new tool or waiting for the tool maintainer to integrate that feature. And because RTMC is plugin driven, the metric data can be provided to your solution in any way you want it. Here is an adhoc example. Lets say you have an end solution that fits your needs to display or make critical decisions based on metric data. But your solution was natively built to communicate via AMQP protocol. This isn't a problem with RTMC. The metric data plugins already provide the metric data, all that is needed is a producer plugin that takes the metric data, via RTMC API, and sends the data to a AMQP server. The AMQP plugin didn't have to gather the metric data, it just has to send the data on that other plugins collect. (RTMC already does provide a AMQP plugin, which this example is based on.) So hopefully you can see the difference between RTMC and other metric tools is versatility.

RTMC Questions:

Q: What are the types of RTMC plugins?

A: Currently, RTMC support three types of plugins; i.e., metric, producer, utility. These three plugin types represent the tasks plugins can perform. Metric being plugins that gather metric data; think of this as the input. An example would be collecting processor metrics. Producer plugins take metric data and perform some action, usually passing the data to some destination; think of this as the output. An example could be storing metric data into a database. Utility plugins perform some action that neither the metric or producer plugins perform. These plugins are general purpose plugins that aid RTMC or add to RTMC feature list. An example is the Plugin_Collector_Utility that RTMC provides. This tool gathers all the metric data from each metric plugin and bundles the data into a single message. That way producer plugins can just get the all the metric data from the Plugin_Collector_Utility instead of querying each metric plugin.

Q: Can plugins do anything they see fit?
A: Yes, RTMC is a multi-threaded plugin engine, so techincally it can do anything you want it to do. It's currently being developed under metric collection, but it doesn't have to be. If you have a need for such an engine to do non-metric collection type of tasks without the need of worrying about multi-thread issues and with plugin support, RTMC will fit that solution.

Q: What is plugin security?
A: For plugins to share and gather data, they must use a RTMC class called RTMC_Plugin. Each plugin has this shared class created for it regardless if the plugin uses that class or not. Having a shared class removes the need for a IPC mechanism to pass data; thus having a shared class creates a lightweight solution over a heavyweight IPC solution. The problem with shared classes between threads is everything that is available to the parent plugin thread is available to other plugins too. You might not want your plugin data shared with all the other plugins, but rather with specific plugins. So a class security base class was added to the RTMC_Plugin class. This allows the plugins to control what methods other plugins can call against the parent plugin shared class. This offers built in ways for plugins to make sure and route data through discovery and protect plugins from other plugins that might not have been written by you, but you still need. Open Source, does not mean it is safe or secure. RTMC allows plugins some extra security if the plugins see fit.

Q: What are the requirements to write a plugin?
A: Plugins are nothing more than a shared library with certain defined symbols that the plugin manager calls. In RTMC's case that symbol is rtmc_plugin_thread. This needs to be a C function that is non-mangled. Meaning if your plugin is a C++ plugin then you need to include the 'extern "C"' tag to the function definition. Now because RTMC is a multi-thread plugin application the function arguements needs to be "void *arg" as the thread call to start the plugin is passed in. Example:

  extern "C" void *rtmc_plugin_thread( void *arg );

{More to be added later.}