Dunning-Kruger Effect

This explains a lot.

The Dunning-Kruger effect describes the phenomenon whereby people who don’t know anything about something think they know more than those who do. The peculiar part is that those who lack the expertise are often far more confident and forceful when they discuss the issue than those who are knowledgeable.

This is because those who are knowledgeable know enough to know what they don’t know, whereas the pseudo-experts do not. The effect ends up being that the idiots with only partial knowledge come off as forceful and confident, while those who are more educated end up hesitating and appearing less versed on the topic.

Here are some main findings:

1. Incompetent individuals tend to overestimate their own level of skill.
2. Incompetent individuals fail to recognize genuine skill in others.
3. Incompetent individuals fail to recognize the extremity of their inadequacy.
4. If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill.

So the question for all of us is simple: how often have we been this guy? How many people do we know who are still this guy? And how do we avoid being this guy in the future? It’s issue when the public is watching two people debate in order to determine who is better qualified in a particular area.

The Project ...

The Project....

The first iteration: Partitioning

In 1946 programmers created software for the ENIAC machine by rewiring plug-boards. Two years later the University of Manchester ’ s Small-Scale Experimental Machine, nicknamed Baby, implemented von Neumann ’ s stored program concept, for the first time supporting a machine language. Assembly language soon became available and flourished. But in 1957 Fortran, the first high level language, debuted and forever changed the nature of programming.

In the 50 years since programming fi rst appeared, software engineering has morphed to something that would be utterly alien to the software developer of 1946. That halfcentury has taught us a few pivotal lessons about building programs. Pundits might argue that the most important might be the elimination of “ gotos, ” the use of objects, or building from patterns.

They ’ d be wrong. The fundamental insight of software engineering is to keep things small. Break big problems into little ones. For instance, we understand beyond a shadow of a doubt the need to minimize function sizes. No one is smart enough to understand, debug, and maintain a 1000-line routine, at least not in an effi cient manner. Consequently, we ’ ve learned to limit our functions to around 50 lines of code. Reams of data prove that restricting functions to a page of code or less reduces bug rates and increases productivity.

But why is partitioning so important?
A person ’ s short-term memory is rather like cache—a tiny cache—actually, one that can hold only 5–9 things before new data flushes the old. Big functions blow the programmer ’ s mental cache. The programmer can no longer totally understand the code; errors proliferate.

The Second Iteration: Productivity Crash

But there ’ s a more insidious problem. Developers working on large systems and subsystems are much less productive than those building tiny applications.

Programmer productivity plummets by an order of magnitude as projects grow in scope! That is, of course, exactly the opposite of what the boss is demanding, usually quite loudly. The growth in communications channels between team members sinks productivity on large projects. A small application, one built entirely by a single developer, requires zero comm channels—it ’ s all in the solo guru ’ s head. Two engineers need only one channel.

The number of communications channels between n engineers is:

n(n-1) / 2

This means that communications among team members grow at a rate similar to the square of the number of developers. Add more people and pretty soon their days are completely consumed with email, reports, meetings, and memos.

Interruptions kill software productivity. Other work has shown it takes the typical developer 15 minutes to get into a state of “ flow, ” where furiously typing fingers create a wide-bandwidth link between the programmer ’ s brain and the computer. Disturb that concentration via an interruption and the link fails. It takes 15 minutes to rebuild that link but, on average, developers are interrupted every 11 minutes. Interrupts are the scourge of big projects.

(click on the image)

A maxim of software engineering is that functions should be strongly cohesive but only weakly coupled. Development teams invariably act in the opposite manner. The large number of communications channels makes the entire group highly coupled. Project knowledge is distributed in many brains. Joe needs information about an API call. Mary is stuck finding a bug in the interface with Bob ’ s driver. Each jumps up and disturbs a concentrating team member, destroying that person ’ s productivity.

Coming up.....
The project in "Embedded Mode" ...

Referenced from:
The Art of Designing Embedded Systems
By Jack Ganssle