Hyperventillating Over Education Rankings


Recently I was at a board dinner where a brilliant young Chinese scholar presented to us on the educational system in China.  I cannot reveal more about which board, his or her name, or anything more about the scholar. Why?

Because he or she still travels to China.  And they were brutally frank about Chinese education – it is riddled with fraud.  Grades are not a function of tests or other objective results, but of bribes, cheating, and influence.

Culture does matter.  I have probably never been more angry than in one particular boardroom in the UK at rudeness directed at me solely for my American accent, but I never questioned the integrity of the data I was getting.  Why?  Because, the other side had an incentive to tell the truth. They knew I would check the objective data in the marketplace and from the UK government’s published statistics.

Nobody in business trusts Chinese, Russian, or Indian data without verification and some reason to believe there is a trustworthy source for the verification.  Now that is a sweeping generalization that does not apply to individuals – I have seen  corrupt Americans and honest Chinese.  But Americans exist in a culture that requires honest data.  Chinese, Russians, and Indians do not.

What does all this have to do with education rankings?

The latest dust up on the quality of American education is based on the results of the Organization for Economic Cooperation and Development (OECD) PISA tests.

These tests are administered in 65 countries to one extent or another and their purpose is defined as:

“… the OECD Programme for International Student Assessment, PISA, has become the world’s premier yardstick for evaluating the quality, equity and efficiency of school systems… By identifying the characteristics of high-performing education systems PISA allows governments and educators to identify effective policies that they can then adapt to their local contexts.”

Based on these results two privileged enclaves in corruption filled China, transparency free Russia, tiny Singapore and Liechtenstein, AND some relevant countries beat the United States in math rankings.  President Obama and many others in Washington are calling on the United States to rethink education at the federal and state level yet again, to adopt the Common Core standards that relate to the PISA methodology, and to  follow the examples of the highly ranked foreign national systems.  But before we once again federalize education to impose overseas methodologies despite the horrendous record of federal intervention (see my favorite chart below), are we really focusing on the right data in PISA?



How are PISA tests graded?

“Each participant has its own group of test correctors, overseen by its National Project Manager.”

Who actually oversees the tests?

“The PISA National Project Managers: Working with the OECD Secretariat, the PISA Governing Board and the international contractor, the PISA National Project Managers oversee the implementation of PISA in each participating country/economy. The PISA National Project Managers are appointed by their governments.” (emphasis added).

So to believe the Chinese results, you have to believe that the data in control of a National Project Manager appointed by the Chinese Communist Party is accurate.  Some authors have stated the technical process of PISA guards against cheating.  The technical appendix reveals no independent auditing or any other control on local country manipulation of the data.

So, are American students falling behind Chinese and Russian students in math and are they a model for the United States in teaching math?  How can we know?

There is of course plenty of evidence in the PISA results to concern any American.  If you superimpose Transparency International’s Corruption Perception Index on the PISA math results and strip out countries who fall in the bottom three quartiles of corruption there are plenty of countries generating results that outperform the United States.  If you then filter all countries below 30 million in population (10% of the US population) under the theory that scale does matter and lessons from large countries matter the most the results are quite different.

We should not be paying attention to China, South Korea, Singapore, Finland, or Liechtenstein, but rather:

  1. Japan;
  2. Canada;
  3. Poland;
  4. Germany;
  5. France;
  6. United Kingdom;
  7. Italy;
  8. Spain; and
  9. United States.

PISA Results Filtered (the overall chart was too big for the post, but click through to see how the filters change the picture).  Not a surprising picture – large capitalist democracies produce real data and scaleable solutions we can study profitably.

Yes, we need to study other education models, but let us not waste time advocating authoritarian educational systems anymore than we would their political systems.




Please enter your comment!
Please enter your name here