Saturday, October 26, 2013

Annals of psychometry: adult cognitive skills by country

You may have read (NYTimes; enjoy the 500 expert comments!) about the recent OECD study of adult skills, which showed Americans lagging behind most other advanced countries. The outcomes more or less recapitulate the PISA results, which are produced by the same organization, led by Andreas Schleicher (Without data, you are just another person with an opinion; Schleicher was trained in physics before moving into educational assessment). Studying adults rather than students makes generational differences apparent: young Koreans scored near the top in the world, whereas older Koreans, who grew up in a much poorer country, do less well. This was also true for Finns, but countries like the UK, US and Australia showed almost no difference between the young and old. It would appear that the Flynn Effect has abated for the current adult population in those countries.

The most striking findings in the report to someone already familiar with this kind of country level data, are that:

(I) Job growth in areas requiring superior ability is high, whereas there is almost no growth in other sectors (see figure below and figure 1.3 in the report). The biggest decline in jobs is in manufacturing-related areas, which are characterized by low, but not the lowest, scores. The number of menial jobs, characterized by the lowest scores, held their own. In other words, in developed countries jobs for factory workers declined while maids, gardeners and servants held their own... a grim future of income inequality based on cognitive ability?

(Difference between mean scores of skilled and "elementary" occupations is about 1 population SD -- Table A3.19. This is about the same as mean score difference between college graduates and those that did not reach upper secondary education! Table A3.9 (l))

(II) Scores on numeracy and literacy showed high correlations: 0.8 -- 0.9 in every country. This is much higher than, for instance, M and V correlation on the SAT.




Here are national score ranges for math skills:


This figure shows that Japanese high school graduates have better literacy skills than Italian college graduates. There is plenty more data like this. Interestingly the standard deviations in scores for Japan and Korea are on the low end compared to other countries.


Roughly every fifth Finn and Japanese reads at high levels (Level 4 or 5 on the Survey of Adult Skills). This means, for example, that they can perform multiple-step operations to integrate, interpret, or synthesise information from complex or lengthy texts that involve conditional and/or competing information; and they can make complex inferences and appropriately apply background knowledge as well as interpret or evaluate subtle truth claims or arguments. They are also good at numbers: they can analyse and engage in complex reasoning about quantities and data, statistics and chance, spatial relationships, change, proportions and formulae; perform tasks involving multiple steps and select appropriate problem-solving strategies and processes; and understand arguments and communicate well-reasoned explanations for answers or choices.
How does someone who can't do these things obtain a good college degree? If only 20% of Finns and Japanese can do these things (the largest fraction among all nations), what fraction of US students are ready for serious college work?

31 comments:

  1. ben_g2:03 PM

    Are the generational differences due to improvements in test taking ability or increases in g?

    ReplyDelete
  2. oregonlocal2:47 PM

    Literacy rates and nutrition (as indicated by average height) for the denizens of industrialized countries have pretty much topped out three generations after WWII. Isn't this sufficient to explain the Flynn Effect?

    ReplyDelete
  3. oregonlocal2:48 PM

    Define "good" college degree. Provide results broken down by ethnicity. The U.S. always scores lower than other industrialized countries on these indicators of health, wealth, and intelligence but we have huge numbers of mestizos and Negroes negatively affecting the mean averages. When non-hispanic whites are broken out separately, the US (unsurprisingly) usually does about as well as the rest of Northern Europe.

    ReplyDelete
  4. In the US height topped out already with those born in the early fifties.

    ReplyDelete
  5. This is probably the 2nd dozenth time i've seen this study blogged and each time I reply is that the USA has a much bigger population than all those countries so not only do we produce a lot of dull kids but on an absolute basis more brilliant ones, too. If you selected only the US students with IQ >110 or some threshold we would rank #1. So we we have 100m students that' still 2 million potential MENSA members or 50% greater than the total population of one of the countries in the study, Estonia.

    ReplyDelete
  6. Richard Seiter5:56 PM

    Do you have actual numbers to substantiate this? You may be right in comparison to small countries, but I suspect you are from the mark if you compare the US to China.

    ReplyDelete
  7. china is not an OECD country so it wasn't included in the study
    assuming the established distribution of IQ scores and the tendency of Asians to score as well if not better than whiles, China is indeed a contender

    ReplyDelete
  8. dxie488:55 PM

    Dr. Michio Kaku has a different opinion,

    http://www.youtube.com/watch?v=NK0Y9j_CGgM

    ReplyDelete
  9. Diogenes9:47 PM

    the nutrition explanation for the flynn effect is believed by the same people who believe in bigfoot and the single bullet theory.

    they apparently don't know that the most g-loaded subtests of the wechsler are also the least affected by flynn.

    they are the people who believe that illiterate savages score lower on iq tests because they are genetically inferior.

    ReplyDelete
  10. Diogenes9:50 PM

    "have better literacy skills than Italian college graduates"


    yes but the italians were tested in japanese.


    given the extremely low and adolescent quality of japanese literature and that japanese adults read comic books and that japanese is the most complex writing system i can discount the entire study.

    ReplyDelete
  11. I am in the process of teaching myself some advanced statistics that I forgot or never learned in engineering and medical school, but just doing some multiple linear regression on state SAT data shows that family income is not a significant predictor of state SAT scores when race and parents’ education are included as dependent variables. I previously found that both Asian Americans and predominantly Asian foreign students have had relatively equal scores with rapid, concurrent improvement. Since foreign Asians would presumably have less material support on average than Asian Americans, perhaps Korea’s improvement has less to do with overcoming poverty than the spread or intensification of a global Asian culture of education. Including variables for income or education plus state participation rates shows a slight decline of scores since 1998 rather than a Flynn effect, but the association between parents’ education and scores has increased over time. That could support a college-for-everybody ideology, even if it does not necessarily follow.

    ReplyDelete
  12. templeruins7:36 AM

    This data would seem skewed. You must take into account the very different educational systems in each country, otherwise this is just pointless number gathering. In UK for example (US too?), you give up Maths at 16, yet in Japan, you don't have the choice, you have to keep learning Maths until 18. In Italy for example, the level of Maths taught is far higher and more in depth that the Math taught to equivalent ages in the UK, yet the UK is significantly higher in this table. In UK the more basic Math tuition compensates for those who struggle with the more difficult Math, whereas in Italy, you are left behind if you can't keep up, even though they are being taught better level of Math. An average Italian student will certainly be more proficient than the average UK student.


    So surely when taking into account the number proficiancy of University graduates in different countries, you must take into account when degree they are doing, and whether they are from a country that makes you continue compulsory Math until 18, or 16.

    ReplyDelete
  13. James Thompson7:40 AM

    http://drjamesthompson.blogspot.co.uk/2013/10/how-illiterate-is-oecd.html I took a critical look at this "lump of a report" but you are right to identify that, although the report dare not use the word, intelligence is realigning occupations, driven by globalization. Any country with bright people and effective educational systems will do well, capturing a good share of "smart fraction" jobs worldwide. Any country with sizeable recent immigrant flows needs to look at those students separately in order to understand the effectiveness of their educational systems. PISA scores show that immigrant scores rise somewhat in the second generation, but do not reach local levels.

    ReplyDelete
  14. The main purpose was to assess adults (e.g., age range up to 60+), so most of the individuals tested have been out of school for many years. Note the Japan-Italy comparison made in the report is on measures of literacy (reading ability and reasoning).

    ReplyDelete
  15. templeruins10:07 AM

    Yes, and I think the most interesting thing was, in this wide age range, those who have been out of schooling for the longest tested with higher scores.

    ReplyDelete
  16. oregonlocal7:26 PM

    Define genetically inferior. Blacks are amply suited to their environment where without strict use of anti-biotics and a punctilious diet sub-Saharan Africa remains "The White Man's Graveyard." Not to mention that a handful of East African tribes (and those descended from them) produce an outright majority of recent long distance running records.


    I referred to both nutrition *and* literacy both of which remain plausible explanations of many aspects of the Flynn Effect. I would further add other media in addition to the written word such as TV, movies, radio, and cell phones to the literacy category due to their deep penetration into even illiterate territories of the globe. A McLuhan corollary if you will.

    ReplyDelete
  17. oregonlocal7:33 PM

    Yes, after experiencing both depression and war while growing up, the boomer's parents did not skimp on making sure that their children did not lack for food or mental stimulation.

    ReplyDelete
  18. oregonlocal7:38 PM

    "Dr. Michio Kaku has a different opinion"



    What a crock. For every "genius" H-1B graduate student there are at least ten Hindu programmers with joke "Bachelor" degrees from some Indian tech diploma mill who force down our wages here in the States.

    ReplyDelete
  19. on the topic of high V, low M

    http://amptoons.com/blog/2005/12/11/the-boy-crisis-part-2-boy-brains-and-girl-brains/comment-page-1/#comment-122790

    ReplyDelete
  20. Dan Gonzalez2:14 AM

    Michio Kaku is a charlatan on this issue. H1b-visas are not for geniuses, O visas are for geniuses with a proven track record of awards/patents, etc..H1B-visas don't even require a 4 year degree in many cases (2 year degrees + experience is common) and are mostly used as serf labor by companies such as Infosys which imports thousands of those indian "geniuses" every year. Also, if there's a shortage of labor/talent in Computer science why are salaries declining for new graduates in that field? http://www.naceweb.org/s09042013/salary-survey-average-starting-class-2013.aspx

    ReplyDelete
  21. Diogenes3:40 AM

    my experience is the mediocre V high M is much more common. i only knew one person, a girl, in hs who scored high V mediocre M. the ashkenazim are superior at both but average or below at S.

    ReplyDelete
  22. Diogenes3:48 AM

    the point was that the nutrition explanation and the testing of people in developing countries are the result of a misunderstanding. and iq test is not a ruler. when the society changes the ruler measures nothing; its measurements are meaningless.

    ReplyDelete
  23. Kennon Gilson8:25 AM

    It would be interesting to see a comparison with all US States and territories.

    ReplyDelete
  24. dxie488:36 PM

    Kaku is a physic professor and he is the protege of Edward Teller, the father of the H bomb. Thus he is aware of those in the highest scientific circle.

    Did not realized that you were competing at that level. Anyway promotion is by merit. Just look at the composition of the top management at Google,

    http://www.google.com/about/company/facts/management/

    Most IT jobs are not location specific. The issue of HiB is not that important. Just look at this case.
    An IT job that paid US$200K+ was secretly and totally outsourced by the employee to China for about US$40K,

    "U.S. programmer outsources own job to China, surfs cat videos"
    http://edition.cnn.com/2013/01/17/business/us-outsource-job-china/

    His daily routine,

    "A forensic image of Bob's workstation revealed his true work habits and typical day:

    9:00 a.m. -- Get to work, surf Reddit, watch cat videos
    11:30 a.m. -- Lunch
    1:00 p.m. -- Ebay
    2:00 p.m or so -- Facebook and LinkedIn
    4:30 p.m. -- Send end-of-day e-mail update to management
    5:00 p.m. -- Go home
    "

    ReplyDelete
  25. how long ago? SAT maths 800 were about a 1000 back in 91.

    http://www.nytimes.com/1992/01/08/us/3d-brother-is-perfect-on-sat-math.html

    ReplyDelete
  26. Diogenes10:34 PM

    the typical natsci professor is as out of touch with reality as a humanities professor.

    ReplyDelete
  27. Diogenes10:34 PM

    '91 and slightly earlier.

    ReplyDelete
  28. "'91 and slightly earlier. there were lots of 700+ M and < 600 V, only one inverted. i think she was ashkenazi, and she was very smart."

    Haven't been able to find percentile ranks for those years, but most likely the inverted scores would not be the same differences in percentiles and SD.
    The ratio of males to females in maths high scores was much higher than in verbal, which was close to 1.

    can't reply to other one due to moderation.

    ReplyDelete
  29. oregonlocal1:36 AM

    Boers are a lot better educated/entertained now as opposed to being culturally deprived Calvinists.

    ReplyDelete
  30. Anonymous11:32 PM

    Why just google's management team? Three of the five current google fellows (singhal, ghemawat, ben gomes) are indians, who started working on h1-bs. Doesn't invalidate your point on the s:n ratio that h1-b holders have, but the O-visa claim is erroneous.

    ReplyDelete
  31. stevesailer6:45 PM

    I recall articles in the 1970s business press where Americans back from studying Toyota and Datsun noted that the Japanese executive salarymen are okay, but that Japan's real strength is in its run-of-the-mill blue collar workers. Japan had the highest low-end in the world, and all these years later, this study supports that.

    ReplyDelete