"Any observed statistical regularity will tend to collapse once pressure is placed upon it for
control purposes." ~ Charles Goodhart
Goodhart's law states that when a feature of the economy is picked as an indicator of the
economy, then it inexorably ceases to function as that indicator because people start to game
it.
Since KEDE reflects the capability of software development organizations in terms of how efficiently they
discover and apply knowledge it
is possible KEDE be used as an input to performance reviews and salary negotiations.
People would say that because KEDE calculation has as its input number of symbols typed per
hour it will be
trivially easy to be gamed by the software developers.
The argument would be that a developer who gets paid per symbol typed can easily earn an
entire year's
salary in a single day without creating any business value whatsoever.
A developer could even copy and paste a huge amount of code, commit it and claim high
productivity.
For instance adding on a regular basis a page of “dead” code that does nothing would gain
0.01 KEDE.
Or maybe 10 pages in order to gain 0.1 KEDE?
Or a developer could commit all the code of an open source framework such as React.
Here are some of the ways such a behavior can be identified:
There are tools on the market for finding dead, unreachable, irrelevant, duplicate code.
There is the practice of code review and many organizations do it.
We calculate KEDE on a daily and weekly basis hence the developer would need to do
the same copy
paste on a daily basis. Does this sound probable?
We calculate KEDE not for a single developer only but for the whole team. If one of
the team members
suddenly is several times more productive than the other wouldn't that be very
suspicious and sound
an alarm?
If individual managers conspire with developers to game their team's KEDE it will be
visible
to the rest of
the company.
We have historical KEDE data for hundreds of thousands of software developers for the
last decade and
some more. If somebody today suddenly starts beating that historical performance
that would sound an
alarm.
KEDE calculation rests on the natural limitation of the maximum typing speed a human
being could
achieve.
That sets a hard ceiling on software development performance.
Some people could say that a whole company could engage in an effort to game their overall KEDE
standing.
Indeed, that is possible, but again how probable is it?
If the company is not brand new then there will be historical data hidden in their
version controlled source code bases.
We can calculate KEDE for all of the company's history.
Then a sudden increase in knowledge discovery efficiency will be easily detected.
In order to show to its clients their KEDE a company will definitely want to have its
records audited by an external independent party.
Same like ISO audits, CMMI appraisals etc.
Such an independent audit will reveal if fraud was committed.
We see that a brute force approach for gaming KEDE is difficult.
What about if developers
adopt subtler
approaches to boost their KEDE?
Some of the ways are:
Moving the same code from one repository to another.
Many organizations do that anyway for reasons not related to KEDE calculation.
Duplicating ode changes into another repository or the same repository.
This can happen when a commit is rebased, amended
or cherry-picked.
Then the new commit has a different commit identifier but contains the same code changes as the original.
Committing say 10 pages of code today.
Then tomorrow delete the 10 pages from yesterday and commit the same 10 pages.
This way at the end of the project the developer would have added only 10 pages of code
but
enjoyed a
positive KEDE throughout.
Such cases should be found automatically before calculating KEDE and KEDEHub does it.