Cost Matrix SLA and ROI
 


SLA
A service-level agreement (SLA) is a part of a service contract where the level of service is formally defined. In practice, the term SLA is sometimes used to refer to the contracted delivery time (of the service or performance). As an example, internet service providers will commonly include service level agreements within the terms of their contracts with customers to define the level(s) of service being sold in plain language terms. In this case the SLA will typically have a technical definition in terms of mean time between failures (MTBF), mean time to repair or mean time to recovery (MTTR); various data rates; throughput; jitter; or similar measurable details.

A service-level agreement is a negotiated agreement between two parties, where one is the customer and the other is the service provider. This can be a legally binding formal or an informal "contract" (for example, internal department relationships). Contracts between the service provider and other third parties are often (incorrectly) called SLAs – because the level of service has been set by the (principal) customer, there can be no "agreement" between third parties; these agreements are simply a "contract." Operational-level agreements or OLAs, however, may be used by internal groups to support SLAs.

The SLA records a common understanding about services, priorities, responsibilities, guarantees, and warranties. Each area of service scope should have the "level of service" defined. The SLA may specify the levels of availability, serviceability, performance, operation, or other attributes of the service, such as billing. The "level of service" can also be specified as "target" and "minimum," which allows customers to be informed what to expect (the minimum), while providing a measurable (average) target value that shows the level of organization performance. In some contracts, penalties may be agreed upon in the case of non-compliance of the SLA (but see "internal" customers below). It is important to note that the "agreement" relates to the services the customer receives, and not how the service provider delivers that service.

Common metrics
Service level agreements can contain numerous service performance metrics with corresponding service level objectives. A common case in IT service management is a call center or service desk.

In the days of operational transaction-processing systems, a service level agreement (SLA) was the agreement between the users and the IT department governing the expectations of the online environment. SLAs were created as a means of measuring and managing the service levels delivered to the end users for that environment. This agreement was a formal statement regarding proper service levels. Although it was somewhat like a contract, it was never a legally enforceable instrument.

SLAs covered two aspects of the online environment: response time and availability. Some SLAs were elaborate; some SLAs were very simple. A typical SLA would be:

  • Response time – 95% of all transactions will be executed in less than 4 seconds.
  • Availability – the system will be up and available 99% of the time between the hours of 8:00 a.m. and 6:00 p.m.
     

Now that the world has embraced business intelligence and data warehousing, these environments also need SLAs. However, the SLAs that govern these environments are fundamentally different from the SLAs that govern the operational environment.

Data warehouse velocity refers to the speed at which data moves through the business intelligence/data warehouse environment, from the initial entry into the operational environment, through ETL  (extract, transform and load) and into the data warehouse, and finally to the data mart environment. Data warehouse velocity measures how quickly data becomes available throughout the data warehouse environment.

Note that velocity is a measure of availability. Some data marts are “pull” processes and only pull the data infrequently. Data may not flow for three months, for example. However, if needed, the data could be pulled on a daily basis. Therefore, actual velocity is different from potential velocity. Potential velocity refers to the speed that could be attained if the data warehouse operated on push processes, not pull processes.

The data warehouse SLAs rely on potential velocity. A typical business intelligence/data warehouse SLA would be:

Potential velocity – 48 hours from entry to data mart usage

One aspect of SLAs that remains fairly constant over the online and the business intelligence/data warehouse environment is that of availability. Both environments need a definition of when the system is available. It is response time and velocity that are very different from one environment to another.

Note that the flow of data may be for a data element that changes. The ETL process may alter the data, and the entry into the data mart environment also may alter the data. Therefore, the exact unit of data may not flow through the business intelligence/data warehouse environment at all. Instead, the mutated form of the data flows through the business intelligence/data warehouse environment.

The development of service level agreements for the business intelligence/data warehouse environment enables the IT department to provide a formal set of expectations for the end users.

An SLA of rackspace can be found at -

https://weill.cornell.edu/its/applications/clinical/datawarehouse/sla.html


Cost matrix

If, by default, all misclassifications had equal weights, target values (class labels) that appear less frequently would not be privileged. You might obtain a model that misclassifies these less frequent target values while achieving a very low overall error rate. To improve classification decision trees and to get better models with such 'skewed data', the Tree heuristic automatically generates an appropriate cost matrix to balance the distribution of class labels when a decision tree is trained. You can also manually adjust the cost matrix.

A cost matrix (error matrix) is also useful when specific classification errors are more severe than others. The Classification mining function tries to avoid classification errors with a high error weight. The trade-off of avoiding 'expensive' classification errors is an increased number of 'cheap' classification errors. Thus, the number of errors increases while the cost of the errors decreases in comparison with the same classification without a cost matrix. Weights specified must be greater than or equal to zero. The default weight is 1. The cost matrix diagonal must be zero.

You can assign error weights to misclassifications by specifying a cost matrix. The following table shows an example of a cost matrix. In this example, the following class labels are used:
  • High risk
  • Low risk
  • Safe
The error weight for classifying customer data as Safe, when it is actually High risk, is 7.0. The misclassification of Low risk as Safe has only an error weight of 3.0.

Table 1. Sample cost matrix table
Actual predicted weight
High risk Safe 7.0
Low risk Safe 3.0
Example:

Your input data might contain information about customers. 99% of these customers are satisfied, and 1% of these customers are not satisfied. You might want to build a model that predicts whether a customer is satisfied by using only a small training set of data. If you use only a small set of training data, you might obtain a degenerated pruned tree. This tree might consist only of one node that predicts that all of the customers are satisfied. This model seems to be of high quality because the error rate is very low (1%). However, to understand which attribute values describe a customer who is not satisfied, a different behavior is required.

You might want to enforce that the misclassification of a customer who is not satisfied is considered ten times as expensive as the misclassification of a customer who is satisfied.


ROI

Return on investment (ROI) is one way of considering profits in relation to capital invested. Return on assets (ROA), return on net assets (RONA), return on capital (ROC) and return on invested capital (ROIC) are similar measures with variations on how 'investment' is defined.

Marketing not only influences net profits but also can affect investment levels too. New plants and equipment, inventories, and accounts receivable are three of the main categories of investments that can be affected by marketing decisions.

Purpose
The purpose of the "return on investment" metric is to measure per period rates of return on dollars invested in an economic entity. ROI and related metrics (ROA, ROC, RONA and ROIC) provide a snapshot of profitability adjusted for the size of the investment assets tied up in the enterprise. Marketing decisions have obvious potential connection to the numerator of ROI (profits), but these same decisions often influence assets usage and capital requirements (for example, receivables and inventories). Marketers should understand the position of their company and the returns expected. ROI is often compared to expected (or required) rates of return on dollars invested.

Calculation
For a single-period review just divide the return (net profit) by the resources that were committed (investment):

    return on investment (%) = Net profit ($) / Investment ($) × 100

or

    return on investment = (gain from investment - cost of investment) / cost of investment