Actionable Analytics Practical Analytics for Practical People

Who is the best – Sachin, Lara, Mark Waugh or Jayasuriya?

A favorite topic that journalists, newspaper column analysts, critics like to discuss is “Who is the best batsman?” Is it Sachin or Lara or Mark Waugh or Jayasuriya or someone else? Two things that crop to my mind is

• Firstly, can we compute the performances of players?
• Secondly, can we compare the computed performances for different players?

These two questions are pertinent as there is no one single metric that can truly compare across different batsmen as they have different strike rates, different aggregates, belong to different teams, play in different circumstances, played under different ground conditions etc. They have contributed to their respective teams differently – changing the momentum by scoring heavily, steadying the team’s position by staying longer at the crease, contributing with the fielding/bowling in addition to batting etc.

In short the questions are

• Can we compare the performance of different individuals who take in multiple inputs in different proportions and produce multiple outputs in different proportions?
• From the performance calculations, can we find individuals in whom improvements can be made? And if yes, to what extent can they be implemented?

In the absence of any clearly defined engineering formula relating inputs to outputs, the problem essentially becomes an empirical issue. A nonparametric approach is provided by the method of Data Envelopment Analysis (DEA).

To answer the above questions, Let us first take a single input - single output scenario as shown below

The Plot for Performance of players is as shown below

The table below explains two approaches to compare performance namely - central tendency approach & extreme point approach with the help of above figure

Data Envelopment Analysis (DEA) is a relatively new “data oriented” approach for evaluating the performance of a set of peer units called Decision Making Units (DMUs) which convert multiple inputs into multiple outputs. Unlike the central tendency approach, DEA is an extreme point method and compares each DMU with only the "best" DMUs (in the above example each player’s performace is compared with the performance of Flintoff).

A fundamental assumption behind an extreme point method is that if a given DMU - A, is capable of producing Y (A) units of output with X (A) inputs, then other DMUs should also be able to do the same if they were to operate efficiently. The heart of the analysis lies in finding the "best" DMU for each DMU. If the "best" DMU is better than the original DMU by either making more output with the same input or making the same output with less input then the original DMU is inefficient. The procedure of finding the best DMU can be formulated as a linear program. The comparison of the actual output produced with the benchmark quantity (quantity produced by the Best DMU) yields a measure of technical efficiency. This is different from economic efficiency, in which one compares the profit resulting from the actual input–output bundle with the maximum profit possible.

In the above example, compared with the best player Flintoff, the other players are inefficient. We can measure the efficiency of others relative to Flintoff by

The relative performance of players in the example discussed above is as shown below

The figure below shows how to make the inefficient players efficient i.e. how to move them up to the efficient frontier. For example, Ponting (as shown in the figure below) can improve in several ways. One is achieved by reducing the input (number of balls) to P1 on the efficient frontier. Another is achieved by raising the output (Runs) up to P2. Any point on the line segment P1-P2 offers a chance to effect the improvements in a manner which assumes that the input should not be increased and the output should not be decreased in making the player efficient.

Thus using DEA, we can compare the performance of Sachin, Lara, Mark Waugh, Jayasuriya, Saeed Anwar over different inputs like No. of balls played, No. of matches palyed, Average ranking of the opposition etc against the outputs like Strike rate, Average, No. of Not outs, No. of victories, No. of Man of the match awards etc.

Strengths of DEA*

• DEA can handle multiple input and multiple output models.
• It doesn't require an assumption of a functional form relating inputs to outputs.
• DMUs are directly compared against a peer or combination of peers.
• Inputs and outputs can have very different units.

Limitations of DEA*

• Since DEA is an extreme point technique, noise (even symmetrical noise with zero mean) such as measurement error can cause significant problems.
• DEA is good at estimating "relative" efficiency of a DMU but it converges very slowly to "absolute" efficiency. In other words, it can tell you how well you are doing compared to your peers but not compared to a "theoretical maximum."
• Since DEA is a nonparametric technique, statistical hypothesis tests are difficult and are the focus of ongoing research.
• Since a standard formulation of DEA creates a separate linear program for each DMU, large problems can be computationally intensive.

Though there are a few limitations in applying the DEA technique, it seems to be one of the most useful and powerful techniques using which we can compare the performance of politicians, film actors, universities, business firms, countries,  regions etc. where every individual unit converts  multiple inputs into multiple outputs.

(* Sourced from http://www.emp.pdx.edu/dea/homedea.html)